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PREFACE 


'The purpose of this monograph is to determine the kind of 
problems occurring in the behavioral sciences, which can be 
solved either completely or partly by factor analysis, and how 
this can best be accomplished. 

As a background to our study, we shall first discuss the charac- 
teristics of the factor analytical models and the assumptions on 
which they are based. That factor analysis has a wide field of 
application as a descriptive condensing aid would appear to be 
indisputable. There are differencies of opinion, however, about 
the possibility of finding factors which can be included in explan- 
atory models. This is partly due to the fact that the meaning of 
explanatory factor analysis is not clear. Consequently an attempt 
will be made to find a meaning for this kind of analysis. We 
shall discuss, from this point of view, the question of how to 
conduct an explanatory factor analysis, particularly when the 
principles of simple structure are applied. 

'The theory of factor analysis embraces a number of compre- 
hensive and complex problems, e.g., of a statistical nature. Some 
of these problems still require to be solved. This monograph will, 
however, be concerned with the questions which are essential to 
the main problem, and will disregard a number of important 
but secondary problems. The monograph is not intended to 
cover the discussion of the whole field of factor analysis. Nor 
will the comments and criticisms, which have already appeared 
and been refuted in psychological and statistical literature, be: 
considered here. 

The monograph mainly deals with questions of current interest 
in order to clarify them, to discuss the problems yet unsolved, 
to describe the attempts that have been made to solve them, 
and to indicate along which lines the solutions are likely to be 
found. 


The problems dealt with are related to various sciences, such 
as mathematics, statistics, psychology, and philosophy. From 
the viewpoint of each of these branches of science, this account is 
perhaps incomplete and imperfect and cannot claim to be “water- 
tight". An attempt is made to integrate the results obtained in 
different special sciences by investigating how factor analytical 
methods can be used to solve problems in the behavioral sciences. 
In any case a useful purpose may be served if someone, who is 
fairly well acquainted with the sciences referred to above, can 
formulate problems and suggest solutions in such a way, that 
scientists, who specialize in a more limited field, will obtain 
a better position from which they can attack some of these 
problems. 

It is obviously impossible, within the scope of this monograph, 
to give a complete account of factor analytical theory. Only 
the background which is essential for the discussion can be given. 
For a fuller account, the reader is referred to classical handbooks 
such as Thurstone’s Multiple Factor Analysis or Thomson’s 
Factorial Analysis of Human Ability or Holzinger & Harman, 
Factor Analysis. 

It has been found expedient to devote the greater part of this 
monograph to a purely mathematical treatment of factor analyt- ‘ 
ical theory, and not to make any attempt to go fully into questions | 
connected with sampling fluctuations. This aspect of the problem | 
is discussed in the latter part of this study. This does not at all 
imply that the statistical viewpoint is of less importance, but is 
due to heuristical reasons. The monograph will be clearer and 
more readily understandable if the mathematical theory and its 
significance are presented at the outset without the complica- 
tions which are inevitable, if sampling fluctuations are included, It 
should be pointed out however, that since the main aim of this 
monograph is to consider how factor analysis can solve problems 
in the behavioral sciences, the treatment of the purely statistical 
discussion is limited to what is necessary in order to fulfil this 
purpose. 

During the time—since 1948 —that I have devoted myself 
to the study of factor analytical problems, personal contacts | 
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have been established with a number of scientists working in 
this field. I have worked under Thomson in Edinburgh, Thurstone 
in Chicago, and Wold in Uppsala. When I was abroad, and also 
on other occasions, I have had the opportunity of personal 
discussions with Ahmavaara, Bargmann, M. S. Bartlett, Burt, 
Cronbach, Eysenck, J. W. French, Holzinger, Lawley, Lubin, 
Peel, Saunder, Tucker, Vernon, Whittle and other investigators. 
During our discussions some of these scientists have expressed 
ideas and points of view, which are not always to be found in 
their published works. 

Professors Husén, Malmquist and Wold have read different 
sections of the manuscript and made suggestions and comments. 
It should be pointed out, however, that I am personally respons- 
ible for the statements made and the results given. 


From Statens Psykologisk-Pedagogiska Institut, Statens Sam- 
hälls- och Ráttsvetenskapliga forskningsråd, and Swerige- Amerika 
Stiftelsen I have received grants for travelling, and for assistance 
with calculation, typing and translation. 

Many persons and institutions—besides those mentioned 
above—have helped me in various ways in the writing of this 
monograph. They are so numerous that it is impossible to name 
them all here. The gratitude I feel towards all of them—from 

. my daughters Mari and Kersti to my teacher and friend Louis 
Leon 'Thurstone—is none the less great. 


Stockholm, March 1957 
Sten Henrysson 


CONTENTS 


PREFACE . 
CONTENTS . š 
SYMBOLS AND "TERMINOLOGY 


I. Tue Basic THEORY or Factor ANALYSIS 


The Rise of Factor Analysis 

‘The Basic Equations. "En 
Component Analysis and Factor Analysis 
'The Geometrical T'heory and Rotation . 
Assumptions about the Nature of the Data . 


П. THe Most Important FACTOR MODELS 


The Need for Restrictive Assumptions . 
The Forerunners of Factor Analysis . 
The Spearman School . i 
Burt's Factor Analytical Methods . 
'Thomson's Contribution . 

The Thurstone School . 


III. DIFFERENT APPLICATIONS OF FACTOR ANALYSIS 


The Problem of Finding the Number of Dimensions 
'The Problem of Finding Descriptive Factors 
Descriptive Factor Analysis and Its Applications. 
Prediction and Factor Analysis & 
Differential Prediction and Factor Analysis è 
Conclusions about Factor Tests as Predictors . 

The Problem of Finding Explanatory Factors . 


IV. FACTOR ANALYSIS AND THEORY OF SCIENCE 


Description and Explanation in the Theory of Science . 
Intervening Variables and Hypothetical Constructs. 


The Distinction between Descriptive and Explanatory Factor 


Analysis . 


ou 


10 


V. ExPLaNATORY Facron ANALYSIS 


The Testing and the Generating of Hypotheses 


br s ow x og 89 

Hypothesis-Generating Factor Analysis and the Principle of 
ТРГУ н, 91 
VI. Factor ANALYSIS ACCORDING TO THE PRINCIPLES oF SIMPLE 
STRUCTURE 
The Meaning of Simple Structure... буа к, уу 
Оби ецца, a a . 100 
"he Selection of "Pest We онен ае 101 
bier Pa ОТР 104 
Analytical Methods for Rotation to Simple Structure... | | 108 
VII. THE PROBLEM or INVARIANCE 

The Concept of "ics EE E E 111 
Methods for Minding Invariante ыз; з, „лу, о, 115 
Simultaneous Rotation to Invariant Solutions. | | | | | 5 7 € 118 
Conclusions about the Methods of Finding Invariance , |. | | 121 
Nonfactorial EUM Исе Б ree ае 122 


VIII. Facron ANALYSIS FROM А Statistica, Pont or View 


The Need of Statistically Acceptable Methods... . 2... 124 


Maximum Likelihood Solutions and Least Squares Solutions . . 127 
Tests of Significance f 


or Factor Loadings ©... 1... 137 
ССОРЕ 139 
ОИ ЧОРТОН 143 
REFERENCES $ 


11 


SYMBOLS AND TERMINOLOGY 


Although certain sections require a special set of symbols, generally 
the following notations and terminology are used in this monograph. 


raw score in test j for individual 7 
subscript for individual 
two subscripts designating tests 


total number of persons in a group 
mean 

standard deviation 

standard score in test j for individual 7 
correlation between test j апа? 

common factor score in the kth common factor for person i 
specific factor score in test j for person 7 
error factor score in test j for person i 
unique factor score in test j for person i 
loading of factor / in test 7 
communality of the jth test 


matrix containing standard scores 

complete correlation matrix with unities in the diagonal 
matrix containing all loadings 

matrix containing all factor scores 

reduced correlation matrix with communalities in the diagonal 
matrix containing common factor loadings 

matrix containing common factor scores 

diagonal matrix containing specific parts of test variances 
diagonal matrix containing specific loadings 

matrix containing specific factor scores 

diagonal matrix containing error parts of test variances 
diagonal matrix containing error factor loadings 

matrix containing error factor scores 

diagonal matrix containing unique parts of test variances 
diagonal matrix containing unique factor loadings 
matrix containing unique factor scores 

matrix containing correlations between all factors 
matrix containing correlations between common factors 
diagonal matrix containing communalities 

matrix containing common factor loadings before rotation 
transformation matrix 

length of test vectors j апа? 

angle between test vectors j and t 


| 


| 
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CHAPTER I 


THE BASIC THEORY OF FACTOR ANALYSIS 


The Rise of Factor Analysis 


Factor analysis has its roots in the psychological and biolog- 
ical discussions around Spencer, Galton, McDougall et al., 
during the later part of the nineteenth century. It was Spear- 
man, however, who started the growth of factor analysis through 
a paper in 1904 on the nature of intelligence. In this paper he 
described a factor analytical investigation of various cognitive 
tests. He started out from a psychological theory which he tried 
to verify with the help of a factor analytical method. The basic 
idea in this method was to trace back the relations between suit- 
ably chosen cognitive variables to a general ability factor—the 
so-called G-factor. Each observed variable was also assumed to 
measure a specific factor not to be found in any of the other 
observed variables. For some decades discussion revolved prin- 
cipally around the validity of this simple model. 

Through Thurstone particularly, development during the 
1930's went beyond Spearman's limited model. More general 
models were developed and, at present, research efforts are 
mostly directed upon these models. 

From the very beginning of factor analysis, an intensive debate 
has been going on about its basis and usefulness. This debate is 
still continuing and is characterized by the fact that the various 
factor analysts have different views as to the value of the methods. 
Even the same investigator's standpoint may vary. The present 
methodological development is characterized by working with 
different special problems connected with the various methods 
or with the general theories, and by seeking new fields of 
application. Since Thurstone’s book, Multiple Factor Analysis 
(1947), and the third edition of Thomson’s book, Factorial Ana- 
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lysis of Human Ability (1948), were published, no general texts, 
more important from a methodical viewpoint, have come out. 
Thus, Vernon's The Structure of Human Abilities (1950 а) is 
mainly of interest as to the application of the factor analysis. 
Spearman-Jones’s book Human Ability (1950) added scarcely 
anything new to the discussions in Spearman’s earlier work. 
Cattell’s Factor Analysis (1952) and Fruchter’s Introduction to 
Factor Analysis (1954) include accounts of the factor analytical 
methods and the technical procedures. 

A growing interest in factor analysis has lately been noticed 
among statisticians. However, they have concentrated themselves 
mainly on statistically efficient but unrotated solutions, which 
need to be adapted and further developed in order to be really 
useful in the behavioral sciences. 


The Basic Equations 


Factor analysis supplies methods for reducing a large number 
of observed variables to a lesser number, in some way more 
fundamental variables or, as they are usually called, factors. This 
is usually done through an analysis of intercorrelations between 
the observed ‘Variables. 

In this section of the monograph an account is given of the 
mathematical theory on which factor analysis is based. This 
account is not intended to be complete but will only bring out 
such characteristics of the factor analysis as are of importance 
in the following discussion. A more complete account is given, 
for example, by Holzinger-Harman (1941) or by Thurstone 
(1947). Elmgren (1955), who introduced factor analysis in Swe- 
den, has published a textbook in Swedish on the subject. 

The present monograph is concentrated on analyses of correla- 
tions between observed variables applied on large samples of 
individuals at a certain time, that is, on the so-called R-technique, 
which, at the present time, is the most developed and used 
technique. It would carry too far to go into various other prob- 
lems, such as analysis of correlations between persons. Many of 
the principal questions regarding R-technique are, to a greater 


“ide 


— 
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or lesser degree, of importance also for the other techniques. 
Further details about different techniques have been given by 
Cattell (1952, pp. 88-126). The discussion here is also limited 
to continuous variables having linear relations with each other. 
The treatment of discontinuous and non-linear cases is discussed, 
for example, by Coombs (1952), Lazarsfeld (1950), and Burt 
(1953). 

This chapter contains the mathematical theory. The purely 
statistical discussion concerning factor analysis as a problem 
of estimation and significance has, as mentioned in the preface, 
been reserved to chapter VIII. 

The basis for the theory are the different observations X;; for 
N individuals in n different observed variables. For convenience 
the word “соге” is used in the following instead of observation 
and “test” instead of observed variable. This, of course, does not 
imply that the methods can only be used on test scores. 

Each one of the scores is expressed as the standard score =, 
in each test according to the following formula 


som раі аа-а O 


Bj 
5; 


M, and s, are the mean and standard deviation for the jth test. 
That means that 


1 N T 1 N 3 
M, 2,89 0 апа per р. (2) 


The test scores expressed in standard scores for the N different 
individuals in the z tests can be shown in a test score matrix like 


the following 


"Su Yu 213 8i 
LO # з Sox 
23 732 75s Эзу 
Z= (3) 


Zni Sng Фаз +++ Зам 
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Since the test scores are expressed in standard scores, the cor- 
relation between the tests j and ¢ for the N individuals ac- 
cording to the formula for the product moment correlation will be 


t=- C (4) 


All the intercorrelations between the different variables сап be 
included in the following correlation matrix: 


Tu Tio Tis soe Tin 
Wan Жы Жы wave б 
Ta Ts Tas +++ Tsa 
К, = (5) 
Tui Tre Trg +++ Tan 


The matrix algebraic expression for the product moment 
correlations will then be 


R,= i ZZ. (6) 
'The aim of factor analysis is, expressed in mathematical 
language, to describe the results in the tests as functions of a 
few variables, which from certain standpoints are more “Ғипда- 
mental” or convenient than the tests. These variables searched 
for, are called factors and can be indicated by Fy, Fy, Fy, ... Г,. 
Each one of the different test scores х; is thus regarded as a 
function of the factors according to the general formula z; = 
Е.Е, ЕЁ»... Fy). 
Factor analysis has hitherto practically only dealt with the 
case, when х; is a linear combination of factors as 


Zj =а Ё. +аз Fu +... + а Fei +... Faq Fy (7) 


Fy; is the factor score of individual 7 in the Ath factor, and а 
is the Ath factor loading in test j. Formula (7) can be regarded 
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as a regression equation where х; is the dependent variable, 
whereas the various Fp; and ау refer to q independent variables 
and their respective regression weights. 

The fully developed system of factor equations for n tests 
and N individuals in expressed by 


Z - AF. (8) 


The matrix A includes the loadings of the q factors in the n 
tests. 


dg Hess eoe | 
uy (dass a ee Uy 
A= (9) 
Qj у... а... Ag 
Way eg mas gx 63 Gag 


The matrix F includes the N individuals’ scores in the q 
factors. 


Be hy va ee on 
Wal xx а Re dit 


а. Жок. Жар жз л. How 
The main interest in factor analysis is at present concentrated 


on how to find methods for calculating loadings in matrix А. 
The variance of a test j in terms of factors is found through 


the variance formula applied on equation (7). 


2 — 56055 5. Henrysson 
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> 2% 
°_ 1-1 
= = 
N N y yo 
> Fi > Fà p Fi P 
ау Ea a odd “ү tt а + (11) 


N N 
2 Fy Fa 2 Ка-ы Far 
a = 

+2 (os UR N te + а-а py ) 


=, 


All the factor scores are for convenience assumed to be in 
standard form and consequently all test variances are equal to 
one. 

Formula (11) can thus be written 


а аъ Ба, + а + 


+2 (p атр, в, + Aja- Go Tra- Fa) =1, (12) 


If the factors are uncorrelated, the section within the paren- 
thesis is dropped. The result will then be 


а 
5? = 2,40 =]. (13) 


The determination of the factor loadings in the matrix A is 
made by the use of the information about the test that is found 
in the correlation matrix 


1 
Е,- 922. (14) 
Since Z — AF, the relation will be 
1 jd ET 
R= HAF (AP) =yAFF A’. (15) 


In this relation the matrix 


vy 
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will include the correlations between the factors. 
If the factors are uncorrelated, Ф is = І, i.e. a unit matrix. In 
that case the formula (15) is reduced to 


R,=AA’. (17) 
The formula for a single correlation ту, is 
Tj = 05 ап + Aja ао + b Ay Quo + 
+ ав lia + (ад аз + аң а) Tp pe 
+ (4 а-л) Ata + 4-1) a) T F1) Fg (18) 


If the factors are uncorrelated the single correlation гу, is 
a 
Tj 2, ака (19) 


The meaning of this is that the test results are considered to 
be linear functions of a number of factors according to certain 
regression equations. The regression coefficients—loadings—in 
these equations must be selected in accordance with the correla- 
tions between the tests so that the relation R, =A Ф A’ is ful- 
filled. 

The factor analytical theory given above has treated the case 
when R, is a correlation matrix. With somewhat different matrix 
algebraic formulae, a similar theory can be applied to covariance 
matrices. 

The main problem in factor analysis is to find the loadings ay, 
in the matrix A which fulfil the condition R, = A Ф A’. When 
A and © are found, then F can be estimated. Nothing prevents 
F from being calculated before A, or both being calculated at 
the same time. At the present stage most scientists consider that 
A is of greater interest than F since A indicates which factors 
and how much of each factor, on the average, the test measures 
(Lawley, 1953, p. 35). Estimates of the factor values Fy; for 
each individual would first seem to be of interest at a later stage 
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when a number of, in some sense, fundamental factors have been 
established. The main problem is at present, therefore, to find 
the loadings from the intercorrelations between the tests, i.e. 
to solve A in the relation R, = АФ A’. 

Unfortunately, in the general case, the correlation matrix R, 
does not include a sufficient number of restrictive conditions for 
a unique solution. Therefore further restrictions must be in- 
cluded. In addition, R, is in practice impaired by sampling errors 
and errors of measurement which complicate the estimation. 

A principle common to most factor analytical methods is that 
the number of factors should be small, usually equal to the rank 
r of the correlation matrix. The addition of this condition, 
however, is not sufficient to obtain a unique solution. 

Two major types of factors are distinguished, namely, common 
and unique factors. The first type refers to such factors as are 
loaded in at least two of the tests in the correlation matrix. The 
common factors therefore correspond to those parts of the vari- 
ances that represent the intercorrelations between the various 
tests. The other major type—the unique factors—corresponds 
to that part of the test variance which is not common to this and 
any other test in the battery of tests. The unique factors are there- 
fore uncorrelated with all other tests included in the battery. 
The unique factors are relative to their scope. By changing the 
composition of the battery of tests, one can displace the content 
of the unique factors in each one of the tests. If a new test is 
added which has a factor, which was earlier unique, common 
with another test, then this unique factor is changed into a 
common factor. In the same way, the removal of a test from a 
battery can result in a common factor becoming unique. 

A unique factor can be divided into a specific part and an 
error part. The later corresponds to the unreliability of the test. 
The specific part is consequently equal to the reliable part of 
the unique variance. 

Divided into three types of factors, the factor equation for 
the ith individual in the jth test gets the following expression: 


m 
a= Px Ещ Gs Sy Qe ji (20) 
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Fpi designates common factors, S;, the specific factor, and гд 
the error factor. The matrix algebraic formulation of the above 
equations is 


Z- A,F, 4 A;Fs4-A,F,. (21) 


Am includes all the common factors, А; the special ones, and 
A, the error factors. The matrices with the factor loadings are 
as follows: 


DONE XT. Ge one Aim 
Be Bg xe BR ч Бы 
An = (22) 
D NE. EXTR. » же By 
аз Ane - Ank Anm 
as 
das 
As= djs (23) 
ans 
а, 
а, 
А, = ау (24) 
а, 


ne 


The two later matrices are diagonal matrices which follow the 
formulae 


S.C.E.R.T., West Bengal 


b2- 
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А.А; 4% (25) 
апа A, Al= А2, (26) 


The matrix R, = АФ A’ is composed of the following three cor- 
relation matrices, namely 


R- A54 45, Q7) 
К; = A, (28) 
R,- А2. (29) 


R contains those parts of the correlations referring to common 
factors. The matrix $ contains the correlations between the 
common factors and is of order m. Its relation to Ф is 


D= К A ; (30) 


Iis here a unit matrix of order л. The matrix Ё is a diagonal 
matrix which includes the specific parts of the test variances 
while R, includes the error variances. The correlation matrix 
R, can be written 


R,=R+Rs+R,=AndbAm+As+ Ae (31) 
The unique parts of the variances are in the diagonal matrix 
Ry=A¥,=Rs+R,=A5+ A? (32) 


This implies that the relation between unique factor loadings 
on the one hand and specific and error factors on the other, is 
the following: 

Ay =(A§+ А2). (33) 


Component Analysis and Factor Analysis 


In conjunction with the division in common and unique factors 
one can make a distinction used, among others, by Kendall 
(1950) and Rao (1955). They distinguish between component 
analysis and factor analysis. In component analysis one does not 
maintain the division between common and specific factors and 


m 
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analyzes the whole correlation matrix R, in what is called com- 
ponents, by using ones in the diagonal, ie., the whole test 
variances. 

In this case, the mathematical model is 


эң — dj Fu + а» Fai +--+ 4 ак Fit 4 ae} 
(721,2... £51,2... Ny &—1,2... g). 


(34) 


The aim of component analysis is a transformation of ће л 
number of tests to g number of components that have certain 
suitable characteristics. The components will in this case give 
the same exact descriptions of the individuals as the tests do. 
The individual test results can be obtained as weighted sums of 
component scores and conversely. 

The component loadings ау in the matrix А are calculated 


from the relation 
К, = АФ A. (35) 


'The most common solution is the principal axis solution. It 
traces R, back to uncorrelated components (Ф = Г), which have 
the characteristic that the first of them absorbs as much of the 
variances as possible, the next one as much as possible of the rest, 
and so on. In this way a considerably lesser number of compo- 
nents than the number of tests can often express the main part 
of the variances of R, and make possible estimation of test scores 
close to their correct values. However, in psychological applica- 
tions, one can as a rule count on the fact that the rank in the 
matrix, R, is equal to 7 and that thus g = л components are 
required for a complete representation of the total variances in 
all the tests. 

Another case of component analysis is an analysis of the 
correlation matrix R,, where the variances of the diagonal are 
divided into reliable and unreliable parts, the later corresponding 
to errors of measurement. In this case the mathematical model is 


2, = aja Fii +азЕ +... + ay Fyi t cb ag Fait ај. (36) 
The loadings in this case follow the relation 


R, =R, +R, =4,® A; + Ap. (37) 
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In the matrix R,, the reliable part corresponds to the matrix 
R, = А,Ф А», and the unreliable part to R, = A?. The number 
of loadings to be estimated is д +n. As previously mentioned, 
however, the number of common factors in psychologicalapplica- 
tions are, in practice, equal to л. The total number of factors will 
then be 2z. 

'The other type of analysis, factor analysis, includes specific 
factors as well. The mathematical model is 


Бы P Fy as S, (38) 
or 2,7 2 aj, Fi aj Syt ау, БТЕ (39) 


'The model one selects depends on whether error factors are 
considered or not. Accordingly, the analysis works with correla- 
tion matrices of the type R, =R + Rs or R, — + Rs + R,. 

Like Mosier (1939, p. 33) one can say that one of the steps in 
factor analysis aims at the following: 


In the application of the theory to the factorization of experimentally 
obtained correlation matrices, the problem becomes one of replacing 
the experimental matrix, R,, with unknown diagonal entries, by a 
reduced matrix, Rọ with known diagonal entries and “true” correla- 
tion coefficients. 


'Thus, in factor analysis, the main attention is concentrated 
on the matrix, R = An $ A, This results in that factor analysis 
will work with correlation matrices R which, in the principal 
diagonal, include values corresponding to the common factors of 
each test. 

This means that one relates the correlation matrix, №, = R + 
РК; + R, to т common factors, n specific ones, and л error 
factors. The specific and error factors, however, can be con- 
sidered as z unique factors. In the principal diagonal of matrix R 
one presupposes those values, the so-called communalities, that 
correspond to the share of the respective test variance belonging 
to the common factors. The formula for the communality k} in 
the jth test is 
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m 
2 2 | 1 n е 
hj = = ji. + 2 (Qj авт p p, - + аа-а т? Eno ғ): (40) 
-1 


The communality is thus equal to the total test variance, with 
the omission of the unique part of the variance. An alternative 
way of indicating the communality is therefore 


Аў = 1— (аў + а). (41) 
This relation has its correspondence in the communality matrix 
Н =I—(Rs +R). (42) 


A weakness in many factor analytical methods is that they 
require knowledge in advance of the matrix, H. There is, as 
will be shown later, no satisfactory method of estimating Н 
beforehand. 

The relation R = A,,¢ А, lacks a unique solution even in the 
case when ф = J. Limiting conditions which give unique and at 
the same time meaningful solutions must be included. These 
conditions will be discussed later. In factor analysis it is not 
always possible to make an exact calculation of the factor values 
Ел, for the N different individuals because of the fact that the 
N(m + п) factor values are more numerous than the number of 


test scores (N х n). А 
The unique part of each test variance is included in the matrix 
Ry = Rs + R, = Rı—R. (43) 


If the reliability coefficients, i.e. the values in the matrix, — R,, 
are known and then consequently also R,, the unique variances 
can be divided into specific variance and in error variance ac- 
cording to the formula 


Rs = (Rs R)—R. (44) 


Respective factor matrices can then be calculated according 


to the formulae 
As=R} and A,- Ri (45) 


The number of factors will be т + 27, if this distinction is made. 
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In this monograph we shall mainly dwell on factor analysis. 
As a rule, in psychological applications this method, as dist- 
inguished from the component analysis, gives a considerably 
lesser number of common factors m than the number of tests л. 
In addition, factor analysis has considerably greater possibilities 
than component analysis of finding common factors which are 
invariant in content from investigation to investigation. Ву 
suitable selection of tests, as will be discussed in chapter VII, the 
solution can by factor analysis be found in such a manner that 
the common factors remain unchanged, e.g. when partly different 
tests are used or when the population is changed. On the other 
hand, in component analysis the possibility of getting invariance 
is considerably less, since the components are hybrids between the 
common and unique factors found by factor analysis. The 
content of these components will, therefore, easily be displaced 
after exchange of certain tests for others which have other unique 
factors. Neither has anyone seriously maintained that components 
found by the methods of component analysis now available, are 
invariant after changes of the test battery's composition (Guil- 
ford, 1954, p. 478, and Kendall, 1950, pp. 65-66). 


The Geometrical Theory and Rotation 


With each one of the various factor methods, one expects 
to find a factor pattern fitted with characteristics particular to the 
method concerned. Some of the methods are at present such 
that it is not possible to find the intended factor pattern by a 
direct solution of the equation system 


R-A,44,. (46) 


'Therefore, the analysis must be made step by step. First, a 
preliminary matrix, A), is calculated. It is selected so as to be 
orthogonal, i.e. follows the relation 


R-A,A, (47) 


and if possible has the number of factors equal to the rank in 
the matrix R. The matrix A, is then transformed to a matrix Am- 


TV Ч 
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This transformation, which in factor analytical terminology is 
called rotation, is made with the help of the transformation matrix 
A according to the formula 


A, =~ А, (48) 


The transformation matrix A should be chosen so that Am 
fulfils the relation 
R = 4,94; (49) 


and, if possible, includes the characteristics that the investi- 
gated problem requires. The rotation is often performed graphic- 
ally by the use of geometrical theory. Geometrical models are, 
on the whole, valuable aids in understanding and carrying out 
the factor analytical operations. 


Fig. 1. 


The common part of a test j is then in the geometrical model 
represented by a vector j, the length of which is h,, the square 
root of the communality. The test ¢ corresponds to the vector 
t of the length hi фу is the angle between the vectors in fig. 1. 
The scalar product of the two vectors j and t is %; h; cos фу. The 
scalar product is equal to 7;,, the correlation between the variables 
jand г. In this way опе can demonstrate the correlation matrix R 
as a cluster of vectors in m dimensions. Factor analysis means 
that in such a vector system is included a system of reference 
vectors each with the length of one unit. 'T'he test vector system 
can then be referred to and described in terms of these reference 
vectors. In the case of orthogonal factors, this is done as follows: 
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Fig. 2. 


The common factor variance in the two tests in fig. 2 is to be 
referred to the factors F, and F,. Vector j:s coordinates are aj 
in factor F, and aj, in factor Fy. The coordinates correspond to 
loadings and will follow the formula 


Yit = алар + ава. (50) 


The case of correlated factors is more complicated but in principle 
similar. 

The localizing of the reference vectors can be done in many 
ways. For each correlation matrix there are an unlimited number 
of possible positions for the reference vectors. In other words, 
there is no unique solution of a factor analytical problem from 
a purely mathematical viewpoint, but many different factor 
patterns are possible. As previously mentioned, certain restric- 
tions must be included in А„ in order to get a unique solution. 
The choice of these restrictions is one of the major problems in 
factor analysis. 

Another aspect of the above characteristics of the reference 
vectors is that various solutions, for example bi-factor and cen- 
troid solutions, can be transformed into each other through 
rotation. 
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This rotation means geometrically that one transforms a 
preliminary system of reference vectors A,, established, for 
example, through centroid solution, so that it changes into 
another reference system A,, to which have been added certain 
restrictions. The transformation matrix A contains functions 
of the angles by which the various reference vectors are rotated. 

A new reference system can also be selected so that the factors 
will be correlated—a so-called oblique solution. This shows itself 
in that the coordinate axes F} and F} in oblique rotation, 
as is seen in figure 2, do not form a right angle. 


Assumptions concerning the Nature of the Data 


If the above mathematical theory is to be applied to psycho- 
logical data with meaningful results, certain assumptions must 
be fulfilled, at least approximately. These assumptions have been 
discussed in detail, for example, by Thurstone (1947, рр. 55-58) 
and Wolfle (1940, pp. 2-4). The most important assumptions 
seem to be the following: 

(a) Individual differences in a great number of tests are 
possible to describe as functions of individual differences in a 
limited number of fundamental variables. In other words, the 
differences between individuals in a number of tests reflect an 
underlying structure, so constituted that the behavior tested 
can neither be referred back to only one undifferentiated trait 
found in each situation, nor only to a very great number of fully 
specific traits. 

(b) The tests have different factor structures i.e. the different 
factors contribute in various degrees in the different tests. 

(c) 'The interaction of the factors in the different tests are 
possible to describe approximately through addition. 

(d) The relations between factors reciprocally and between 
tests and factors are approximately linear. 

(e) The same pattern of loadings applies to all individuals in 
the population and the differences between individuals depend 
on different numerical values in factor scores. The loadings state 
the average weight given the factor concerned, while the indi- 
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viduals’ deviations from the average are given by their factor 
scores. 

The validity of the above assumptions is, at least in principle, 
testable by empirical methods. If contrary to the assumption a 
the individual differences in the test could be traced back to one 
undifferentiated trait, then, after correction for attenuation, the 
correlation between the tests should be equal to 1. If the traits 
are completely specific the correlations should be very close to 
zero. In a number of correlational investigations of tests, the 
correlation matrix has proved to include correlations of varying 
sizes and structures. Hence, the validity also of assumption b 
that the test can show different factor structures. No scientist, 
well orientated in factor analysis, doubts the validity of the 
assumptions а and b for a number of data in the behavioral 
sciences. 

On the other hand, there is a dispute about the validity of 
assumption c, i.e. that the factors combine by addition. A com- 
mon objection in psychological applications is that the different 
sides of the personality act together in a dynamic way that 
cannot be descibed by addition. Thurstone, for example, answers 
to this that no one should look for factors which are completely 
isolated from the rest of the personality for the rest. He points 
out that if some structure or relation exists, then it must in some 
way be noticeable and also possible to identify (Thurstone, 
1947, p. 59). Besides, further analyses and investigations will 
have to show whether analysis of factors is possible in a field 
and if the results are meaningful (op. cit., p. 340). However, 
one ought to count on the fact that it is not always possible to 
break down the performance on any test into a sum of simpler 
functions. 

The assumption d refers to the linearity between variables. 
This is also an empirical question that must be decided from case 
to case. Often one can get linearity by the use of a suitable scaling 
procedure, e.g. a general transformation to 'T-scores. If linearity 
exists between the tests, it is often reasonable to assume at least 
approximative linearity for the factors. In any case an analysis 
can be made into a number of descriptive factors, which are 
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linearly connected with each other and the tests. Factor analysis 
should be used in less explored fields where it is necessary to get 
first approximation and for that purpose linear factor equations 
can often be used. Further experiments will then be basis for 
more exact formulations (op. cit., p. 56). 

Even the truth of the assumption e about the generality of the 
factors has been strongly questioned. Many psychologists point 
out that different people can obtain the same score by different 
approaches, i.e. that their results require wholly or partly different 
factors. This implies that the same factor equation cannot be 
applicable to all individuals in a certain population (Allport, 
1945, pp. 242-248). As a matter of principle, this only means 
that the factor pattern becomes more complicated and gives 
place to more factors than those that should be used if all indi- 
viduals applied the same procedures for solution of the task. 
'Тһе risk remains, however, that if several procedures for solu- 
tion, each requiring a very different factor structure, are used, 
then the loadings in all the different factors will be so small and 
so many that they cannot be found by the use of such a crude 
method as factor analysis. Also in this respect the results in each 
particular case determine whether the obtained factor pattern 
is applicable and meaningful for all individuals. 

From the above discussion, it is evident that the factor equation 


m 


y= 2, аъ Fico Gs S; Ge Eji (51) 


is often approximately useful as a descriptive model. Further 
investigations will, in each particular case, determine to what 


degree this is true. 


CHAPTER II 


THE MOST IMPORTANT FACTOR MODELS 


The Need for Restrictive Assumptions 


If the above assumptions are valid for the data investigated, 
then factor analysis can be used in finding a model that describes 
the relations between the tests in the form of a few relatively 
independent factors. There are, as previously mentioned, many 
alternative factor analytical models, which all reproduce the 
correlation structure in a certain set of data. The scientist must, 
therefore, choose which one of all possible models he wants to 
use. This choice is dependent on his requirements on the factor 
structure and is based upon more or less explicit theories about 
the structure. 

In the following pages are discussed some of the most import- 
ant schools of factor analysis and the characteristics of their 
different factor models. This is not an attempt to give a full 
historical account but only to bring out aspects of the different 
schools in factor analysis particularly interesting in this con- 
nection. 


The Forerunners of Factor Analysis 


During the later part of the 19th century there was a debate in 
Britain, and also on the Continent, about the presence of a 
general ability and special abilities in the cognitive field. In this 
debate Spencer and Galton, among others, took part. Spencer 
maintained a theory of general factors that has great similarity 
to Spearman’s two-factor theory, later proposed while Galton 
was also inclined to accept what is called group factors in factor 
analytical terminology. Galton also looked for support from cor- 
relation studies in a manner which foreshadowed factor ana- 
lysis. 
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During the development of the correlation method that took 
place at the turn of the century, Edgeworty and Pearson devel- 
oped the factor equation in its multiple form with orthogonal 
factors. They worked with anthropometrical measurements and 
wanted to condense the essential information in a larger number 
of measurements of different parts of the body to a few independ- 
ent characteristics. This was done through the principal axes 
solution. Since they worked with the whole variance in their 
variables, i.e. with correlation matrices that had ones in the 
diagonal, the number of factors became the same as the number 
of variables. The method is thus a form of component analysis, 
but through this contribution, that came before Spearman's, the 
formulation of the equations of factor analysis had started. 'The 
beginnings of factor analysis has been described, among others, 
by Burt (19492). 


The Spearman School 


It was Spearman (1904) who, through his paper “General 
intelligence objectively determined and measured”, seriously 
put life into the development of the factor analysis. He introduced 
his factor analytical method to support one of the theories of cur- 
rent interest about the nature of cognitive processes. Spearman 
adhered, as previously mentioned, to the monarchic position. 
In Spearman’s formulation it was called the Two-Factor theory, 
meaning that results in cognitive variables are referred to a 
general ability factor that in greater or lesser extent is found 
in all cognitive variables, and in addition, one unique factor in 
each variable. In his paper, Spearman (1904, p. 284) described 
his theory as follows: 


АП branches of intellectual activity have in common one fundamental 
function (or group of functions), whereas the remaining or Specific 
elements of the activity seem in every case to be wholly different from 
that in all the others. 


The factor equation for the test j is 
23 = Ayn Еи + Ay Ол. (52) 


3 — 576055 S. Henrysson 
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Spearman does not make a complete distinction between 
specific and unique factors and often uses the name specific 
factor for both. In certain cases he works with correlations cor- 
rected for attenuation. In this case the specific and unique 
factors in a test coincide since no error factor is included in the 
test variance. 

The above factor analytical model was included as a part of the 
so-called Noegenetic Theory about the cognitive processes, 
which, according to Spearman, was supported by his factor 
analyses (Spearman, 1931, pp. 112-117). Spearman clearly 
understood that his correlation matrices could be referred back 
to other types of factor models, but considered, however, that 
his model was the most comprehensive one and statistically the 
most plausible. In addition, the two-factor theory provides 
simple and unique solutions (Spearman, 1927, pp. vi-vi1). 

'To his death, in 1942, Spearman maintained his conception 
about the dominating importance of the G-factor, which is 
evident from the posthumous book Human Ability published by 
Jones іп 1950. The chapter “Existence of “С?” thus concludes 
with the following statement (p. 61): 


We have marshalled the recent evidence as to whether or not the 
general factor designated as G may be properly said to exist. By this 
is meant that it presents at least the cardinal statistical characters— 
definiteness, comprehensiveness and stability—which are indispensable 
for the purpose of science. The answer, recorded in an immense amount 
of such research as that of Stephenson, Brown, Alexander and Holzinger, 
has been a decided affirmative. 


Spearman does not enter into any detailed discussion about 
what he means by the "existence" of a factor. He states that such 


a question has strong metaphysical elements. In Human Ability 
he and Jones say about the G-factor (p. 52): 


Does such a factor really exist? If so, what is its nature? Taking the 
former question first, let us clear the ground by repudiating all the 
metaphysical implications that some authors have sought to read into 
the terms really "exist". Such topics belong, it is believed by the senior 
writer, not so much to empirical as rather to philosophical psychology, 
with which we need not here trouble ourselves. (He has treated it 
freely elsewhere, as in Psychology Down the Ages.) For us, the question 
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as to the observation of С reduces itself to that of validity of certain 
“hierarchic” equations. We have only to inquire whether or not these 
have been sufficiently well satisfied for the purpose of determining and 
stabilizing the factors at issue. 


The existence of the G-factor will, according to Spearman, 
be dependent on that the correlation matrix has the rank one. 
The two-factor theory is evidently to be considered as a hypo- 
thesis that has to be Statistically verified by analysis of suitably 
selected test batteries. 

In any case, if the rank is one in the correlation matrix R, then 
there is a general factor, that also is invariant under certain 
conditions. The latter is an important requirement for a factorial 
solution, if it is to have scientific value. That a factor is invariant 
means that its content is unchanged even if the design of the 
factor study is changed, for example, by exchange of certain 
tests for others in the battery. A general factor in a matrix of rank 
one will, therefore, be invariant when adding tests measuring 
the same general factor and unique factors. This is evident from 
the following discussion. 

If a correlation matrix R can be described by a general factor 
only, then the equation for general factor loadings а„ is 

a=" (е j,t=1,2...0; е+]+®). (53) 
it 

A new test л +1 has the loading &m+11 in the general factor 
and the correlation 7, ¢n41) with the test e. The squared loading 


“az,” determined with the help of the new test will be 


eq 79 eann, (54) 


el 
утар 
ЕЈ 


When the loadings, under the assumption of one common factor, 
are included, this will give the equation 


а ара, а 
“айр? = е1 2л Ger ЕА (55) 
ад Gc) 


By this, the loadings are shown to be unchanged after addition 
of the new test. 
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The investigation of the G-factor theory will, of course, give 
an incorrect result, if the tests are so selected that some other 
general factor fully or partially replaces Spearman’s G-factor. 
If, for example, all tests are very time-limited, this can result in 
a mixture of Spearman’s G-factor and a speed factor. The 
factor equation, in such a case, will get the following form 


Zj =ал Fu + dj Fy, + ay Uj. (56) 


In the analysis, these two general factors can appear as one. 
Spearman has tried to get away from this weakness by requiring 
that certain key tests, which experience has shown to be highly 
loaded with G, shall be included in the battery. 

Spearman’s factor analysis has the characteristic of testing 
the G-factor hypothesis on a test battery measuring the G-factor, 
to which the new tests are added. If the hypothesis is correct, 
the addition of a new test should not result in a change in the 
factor structure of the old tests measuring G and at the same time 
the two-factor equation 


By =аң Fii +а н (57) 


will be applicable to the new test. 

The ultimate criterion of this is the tetrad criterion by which 
one can verify that the correlation matrix has rank one. In 
practice, it was difficult to get the tetrad criterion fulfilled when 
Spearman and his followers started to expand factor analysis to 
new and more tests (Burt, 1949 а). But Spearman’s defence went 
mainly along two lines. He maintained that the test battery could 
include tests that are too much alike and consequently have 
certain parts of their specific factors in common. "These specific 
factors then manifest themselves in the analysis as group factors, 
i.e. factors that are included in more than one test but not in all 
of the tests in the battery. For that reason he stated that the tests 
that gave rise to such false group factors, so-called “overlapping 
specifics", should be dropped (Spearman, 1927, pp. 150-160). 
In opposition to this, the critics, in particular Thomson, main- 
tained that in such a case the scientist starts out from what he 
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wants to get verified and changes the test battery until the results 
verify the hypothesis (Thomson, 1933). 

The second line of defence was of more statistical nature. 
Spearman considered that the fact that the tetrad criterion was 
not fulfilled could be caused by sampling errors in the correla- 
tions. Therefore, he and Holzinger worked out a formula for the 
sampling distribution of tetrad differences (Spearman, 1927, 
р. X). This formula was a decisive progress, but many misused it. 
They treated individual tetrad differences and investigated 
whether their size was greater than a certain number—for 
example, three—times the standard deviation of the sampling 
distribution. If it was not greater, the tetrad criterion was con- 
sidered to be fulfilled and the two-factor theory as verified. 
Particularly, with a small sample, this was easy to attain. This 
method of proceeding was branded by the critics (Emmett, 
1936), and instead, one started to investigate complete distribu- 
tions of tetrad differences. 

As the requirements for test construction and statistical preci- 
Sion increased, it became more difficult to maintain the two- 
factor theory in its original form. The last major attempt to 
Show its validity was made by Brown and Stephenson (1933). 
They investigated a sample of 300 10-year-old boys with 22 
carefully selected tests. In order to get the tetrad criterion ful- 
filled, they had to eliminate what they considered to be a verbal 
factor and two “false” group factors with the use of partial cor- 
relation. In addition, they removed one test that distorted the 
tetrad criterion. The investigation was exposed to much criticism 
(Thomson, 1933). The debate about this and similar investiga- 
tions and the development of multiple factorial analysis has 
resulted in that the presence of group factors in the cognitive 
field is now generally accepted. Even Spearman had to accept 
them as facts although he tried to minimize their importance 
and was inclined to consider them as products of the environment 
(Spearman, 1927, p. 242). The essential point in Spearman's 
effort, however, is to determine the presence of and the extent of 
the G-factor, and to find good measures for this. He also con- 
Siders that he has obtained proof of the central role of the G- 


38 


factor and that the difference between universal factors and 
non-universal factors is primary and fundamental (Spearman, 
1933, p. 599). 

Spearman's co-worker Holzinger has modified the two-factor 
theory to a so-called bi-factor theory, in which there is also place 
for a group factor in each test. The factor equation, in this 
case, is as follows: 


Sg = ayFy; + Ay Figs + ayy Uj, (k =2, 3,...m * 1). (58) 


Holzinger uses his method primarily for testing hypotheses 
regarding the presence of certain group factors. He builds up 
his test battery so that it contains three or more tests, that can 
identify each of the hypothesized factors (Holzinger-Swineford, 
1939, p. 1). The factor analysis itself takes the form of a verifying 
of the hypotheses, but it can also result in that these are rejected 
and eventually revised for the benefit of another bi-factor solu- 
tion, which then ought to be tried out on new data. 

Holzinger thus tried to refer the correlation matrix R back 
to a pattern that includes in each test, one general factor, one 
group factor common to at least two of the other tests, and one 
for the test unique factor. The group factors can also be fully 
excluded in certain tests. According to Holzinger, this model 
satisfies the requirement of simplicity. The factor structure can; 
for example, look as follows (for the sake of simplicity illustrated 
only by the sign of the common factor loadings): 


TABLE 1. 
Factor 
Test I II TIT IV 
ry 

1 + + 

2 + + 

3 + + 

4 + + 

5 + i 

6 + + 

7 + + 

8 + m 

9 + + 
10 + ds 
11 + 
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All tests are loaded with the general factor I, and tests 1, 2, 
and 3 also with the group factor II, and so forth. Test 11 is 
loaded only in the general factor. 

The general factor loadings are calculated by applying Spear- 
man's tetrad principle to all combinations of tests that do not 
have any common group factor (Holzinger-Harman, 1941, pp. 
116-121). Then a calculation is made of the residual matrix after 
the removal of the contribution of the general factor from the 
matrix R. If the hypotheses regarding the group factors are 
correct, the residual correlations between all tests that are not 
expected to measure the same group factors, should be zero 
within the limits of sampling errors. The next step is to cal- 
culate the factor loadings for the group factors. This is done by 
applying the tetrad principle to each one of the groups of tests 
that is supposed to measure a certain group factor. 

Holzinger is searching for a certain amount of invariance or 
“stability” as he often calls it. This invariance is dependent on 
the tests being so selected that they group themselves into clusters 
of tests, each cluster corresponding to a group factor expected. 
The invariance that Holzinger can expect, is thus very limited. 
If the centroid of a cluster is changed by the addition of new 
tests, then the content of the corresponding group factor is 
also changed. 

The bi-factor solution, however, is invariant, if new tests are 
added to the battery, which have the same common factor struc- 
ture as certain of the earlier tests. This can be shown in an analo- 
gous way as in the above case with only one general factor. The 
invariance is closely connected with the fact that the bi-factor 
solution is a successive application of the tetrad principle. 

An example of the use of the bi-factorial method is given in 
the study The Stability of a Bi-factor Solution published by Hol- 
zinger and Swineford in 1939. In this study, the factors found 
in two samplings from partly different populations correspond 
well with one another and with the hypotheses. However, Swine- 
ford (1949) later tried an alternative bi-factor solution on the 
Same material with success and used this new factor model once 


more on new material. 
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Very few parallel investigations have been made with the bi- 
factor method, and therefore it is difficult to appraise the 
possibilities of the method of giving factors that are invariant 
after great variations have occurred in the tests and popula- 
tions investigated (Cattell, 1952, pp. 145-146). 


Burt's Factor Analytical Methods 


Burt has been in on the factor analytical debate from the 
beginning. In his major work, The Factors of the Mind, published 
in 1940, he has discussed the theoretical basis of factor analysis 
even from the viewpoint of general science and philosophy. In 
his book he treats factor analysis mainly as an aid in finding useful 
descriptive concepts but will not give these any special meaning 
in terms of underlying structure (Burt, 1940, p. 227): 


Rigorously speaking, factors cannot be regarded as substances, or 
as parts of a substance, or even as causal attributes inhering in a sub- 
stance. They are not separated “organs” or “isolated properties" of 
the mind; they are not primary “abilities”, “unitary traits", “mental 


powers or energies". 'T'hey are principles of classification described by 
selective operators. 


He considered the selection of factors as arbitrary in principle, 
and compared the role of the factors with the network of meri- 
dians and parallels of the globe. This network has no meaning 
beyond the descriptive-mathematical, but it is still very useful 
for describing the position of any point on the globe. The same 
is the case with the factor analytical model. It can give psychology 
as well as other sciences a few permanent, and independent terms 
of reference. 

Burt treats factor analysis as a deductive method, and considers 
factors found by this method as systematized descriptions, which 
are the result of certain mathematical manipulations with the 
data. He emphasizes that this deduction is not sufficient to 
establish that the obtained factors are fundamental. 

In later years Burt seems to be more inclined to use factor 


analysis for solving real structure problems if used for testing 
hypotheses. 
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Burt uses factor analysis as an aid in verifying a “hierarchical 
theory" as to the nature of the cognitive processes. This theory 
implies a model containing one general factor and a hierarchy 
of group factors of decreasing importance. Peel (1953) has given 
a review of this method and its application. Burt has also sup- 
ported his theory by data from non-factorial research. In his 
interpretation of these factors he obviously regards them as 
more than descriptive concepts (Burt, 1951, p. 47): 


Moreover, this hierarchical structure appears to be something more 
than a mere logical structure, a mere descriptive device; it would seem to 
be the outcome of a developmental process, resting on a genetic founda- 
tion, and doubtless corresponds to a similar structural organization in 
the nervous system itself. But a concrete interpretation in causal terms, 
like all such interpretations, can never be inferred from tables of correla- 
tions and matrices of factor-measures alone. To establish it we must 
introduce non-statistical evidence at some stage of the argument. 


Burt, in this case, factor analyzes according to hypotheses 
advanced beforehand, which have decided the choice of tests 
(Burt, 1950a; Moursey, 1952). The mathematical analysis starts 
with the calculation of a centroid solution. The hypotheses can 
be such that the centroid solution will get, for example, the 
expression given in table 2. 


TABLE 2. The centroid solution. 


Factor 
Test I II ш IV 


1 
+ 


I 


eo MO t N= 
thet tees 


In any case, when investigations are made in the cognitive 
field, the rotation to positive loadings is made according to the 
hierarchic division that is indicated by the positive and negative 
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TABLE 3. Burt's Group factor solution. 


Factor 
Test I II III IV V VI VII 


+++ 
i 


омол фо н 
HEE HEHH 
thet 

i 


loadings of the centroid solution. Even the negative loadings are 
rotated to positive factors, which, in relation to other factors, 
are orthogonal. This results in an increase in the number of 
factors. The hierarchical structure in such a case can have the 
sign pattern indicated in table 3. Burt considers that a hierar- 
chical structure of this type is preferable, because it is simple 
and corresponds with psychological principles (Burt, 19491). 

In certain cases, for example, when he factorizes personality 
variables, Burt uses the centroid solution without rotation, since 
bipolar solution with both positive and negative loadings in his 
opinion is reasonable in this domain. 

Burt's methods have been criticized by Thurstone, Thomson, 


Cattell, and others, for not giving invariance. Cattell says, for 
example (1952, p. 249): 


A last principle of rotation is not to rotate at all! Such an economical 
procedure has its devotees. The most systematic defence of the proce- 
dure has been made by Burt, followed by Eysench. The only substantial 
claim yet put forward, apart from economy, is that it can be empirically 
shown to give factor invariance just as well as simple structure or any 
other adequate method. On grounds of experience this cannot at the 
moment be adequately proved or disproved, since insufficient repeat 
factorizations of the same variables under adequate experimental and 
sampling conditions exist today. Those which do exist, in the writer’s 
opinion, show more indubitable invariance for simple structure than for 
nonrotation. However, on theoretical grounds, the use of the unrotated 
matrix, involving a first general factor, with no zero loadings, does not 
in the first place agree with the general psychological expectation that 
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all factors will be of the same general species of loading pattern and the 
same general order of mean variance. Further—and this is more surely 
fatal to the notion that such factors can be invariant—one has to re- 
cognize the statistical fact, that, except for one unique situation (when 
the hierarchy of correlation is not upset), the addition of new variables 
to the battery alters the centroid of those preéxisting. Their loadings in 
the first (and in all subsequent) factors vary according to the company 
they keep. This makes the use of unrotated factors psychologically and 
statistically meaningless. 


This critisism of Burt's methods is somewhat exaggerated, 
since Burt in certain cases uses rotation. The important thing, 
however, is that the rotation in these cases is done according to 
the pattern indicated by the position of the negative and positive 
loadings of the centroid solution. 

The centroid method, which is the foundation for Burt's 
solutions, is in itself not invariant. It is well known that centroid 
solutions are changed if new tests are added. This is evident if 
a test is added to the solution in table 2, which has the same 
common factor structure as, for example, the first test. Then 
factors II and III no longer fulfil the requirement in the centroid 
solution that the sum of the loadings of each factor, except the 
first one, is equal to zero, since each of the columns II and III has 
received a positive addition. A new centroid solution should give 
other loadings. Burt does not seem to count on invariance after 
such changes in the test battery, as previously mentioned. But 
on the basis of his earlier studies in the same field, he can select 
the tests so that a new solution becomes invariant in relation to 
the solution obtained in the earlier studies. This seems to be an 
art of balancing the choice of the tests in the battery which 


requires practice and skill. 


Thomson’s Contribution 


Thomson appeared early as Spearman’s foremost critic and 
has in recent years fulfilled the same function concerning Thur- 
stone. Thomson’s contribution to the factor analytical theory 
discussion has been of a predominantly critical nature, but he has 
also contributed to the development of the mathematical theory. 
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In his main factor analytical work Factorial Analysis of Human 
Ability, Thomson, maintains a sceptical attitude concerning the 
scope of the factorial analysis in the solution of structure prob- 
lems. He points out that in natural sciences, such terms аз“епегру”, 
“neutron” and "gene" are considered by scientists as manners of 
speech" intended to characterize “abstract ideas" (Thomson, 
1948, p. 241). Thomson considers that similar terms in psycho- 
logy can be motivated by their usefulness, but emphatically points 
out the danger in giving them any other meaning. He stresses the 
danger in giving the factors (Thomson, 1948, p. 42): 


. any more "'reality" than an average or an index of the cost of living, or 
a standard deviation, or a correlation coefficient—though, on the other 
hand, it may be admitted that some of them, Spearman's g for example, 
may come to have a very real existence in the sense of being both useful 
and influential in the lives of men. 


He has as an alternative model suggested his Sampling theory, 
which means that a very great number of "bonds" sampled in 
various ways by the different tests explain the correlations. 


The Thurstone School 


Thurstone formulated the factor analytical model in a rather 
general form. He made use of matrix algebra and pointed out 
that the least number of common factors m that is needed for 
explaining the correlation matrix Ё, is equal to the rank r of the 
matrix. 


Thurstone strongly separates two different tasks for factor 


analysis. He points out in his major work Multiple Factor Analysis 
(Thurstone, 1947, pp. 503-504): 


A factorial analysis can be made for one of two purposes, namely, 
(1) to condense the test scores by expressing them in terms of a relatively 
small number of linearly independent factors or (2) to discover the 
underlying functional unities which operate to produce the test per- 
formances and to describe the individual differences eventually in terms 
of these distinguishable functions. ... It is the second objective that 
has guided the writer, since 1931, in developing multiple-factor methods. 
It is the first objective that is usually implied by mathematical statisti- 
cians who write on factorial problems. This difference in objectives is 
largely responsible for controversies about factorial method. 
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The main object of Thurstone’s factor analytical investigation 
is thus directed on the clarification of the underlying processes. 
This purpose is, however, linked with a pragmatically-oriented 
attitude in his theory of science. This theory is further developed 
in a chapter of Multiple Factor Analysis called “Оп the nature 
of science", wherein, among other things, he says (op. cit., p. 51): 


It is the faith of all science that an unlimited number of phenomena 
can be comprehended in terms of a limited number of concepts or ideal 
constructs. Without this faith no science could ever have any motiva- 
tion. To deny this faith is to affirm the primary chaos of nature and the 
consequent futility of scientific effort. The constructs in terms of which 
natural phenomena are comprehended are man-made inventions. To 
discover a scientific law is merely to discover that a man-made scheme 
Serves to unify, and thereby to simplify comprehension of a certain class 
of natural phenomena. A scientific law is not to be thought of as having 
ап independent existence which some scientist is fortunate to stumble 
upon. A scientific law is not a part of nature. It is only a way of compre- 
hending nature. 


The selection of a factor analytical model is to a high degree 
for Thurstone a question of convenience. It is a question of 
selecting the simplest and most useful among many models. 
Thurstone’s basis for the selection of model is the principle of 
parsimony. Starting from this principle Thurstone states a 
number of requirements to be fulfilled by the factor model if 
this is to be suitable for expressing the underlying processes. 

Thurstone’s most important requirements are: 


I. Parstmony.—According to Thurstone a general rule in 
scientific theory construction is to use as small a number of 
concepts with as great a scope as possible. Thurstone therefore 
tries to find a factor system, which, for example by being simple, 
may be preferred in the scientific construction of theories. 

The parsimony criterion is also used by Thursone for fulfilling 
another of the requirements he places on the factor model, 


namely invariance. . А . 

П. INvanrANCE.— The factors ought to be invariant in order 
to be used in scientific models of more general range. This means 
that factors should not only be dependent on just the one constel- 
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lation of variables and individuals that are being analyzed at the 
moment, but also appear in analyses under different experi- 
mental circumstances. A verbal factor found in analysis of a 
test battery in one population, for example, should, in order 
to be invariant, also appear in a factor analysis of another po- 
pulation with a fully or partially different test battery. 

There are different types of invariance which are mutually 
related. They are dicussed by Thurstone (1947, pp. 360-376), 
and may be systematized as follows: 

(i) Numerical invariance between analyses of the same battery 
of tests given to several samples from the same population. Different 
independent factor analyses of this kind should give factor load- 
ings which remain the same when allowing for sampling errors. 
This can be achieved by the use of a factor analytical method 
giving unique solution. 

(ii) Configurational invariance found in analyses of the same 
test battery given to several different populations. In this case the 
factors should be the same in all analyses, but the numerical size 
of the loading should be changed according to the differences 
in the composition of the populations. If, for example, a popula- 
tion has considerably less variance in some of the tests, this will 
result in a corresponding reduction in the size of the loadings. 
The numerical values should be changed but the configuration 
of loadings remain the same. This naturally assumes that the 
factor structures in the two populations are really comparable. 
Should many individuals in one population solve the items in 
the test by means of a procedure which is partially different 
from that used by the other population, then the results will 
not be comparable. 

(iii) Numerical invariance between analyses of batteries including 
partly different tests; each battery given to samples from the same 
population. In this case, those tests, which are the same for several 
batteries, will keep their factor loadings unchanged within the 
limits of sampling errors. This presupposes, of course, that the 
variables in both batteries are selected in such a way that the 
common factors are the same. If, for example, a common factor 
is loaded in three tests in one battery, and then only one of these 
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tests is included in a second battery, it is to be expected that the 
common factor becomes specific in this second battery. 

(iv) Configurational invariance between analyses of batteries 
composed of partially different tests, each battery applied to a 
different population. In this case, a test that is included in two 
partially different batteries will retain its Structure, but with 
changes in the size of the loadings equal to the dissimilarity of 
the populations. As in case ii, it is here assumed that the com- 
mon factor variance is similar in both batteries. 

If the requirement of invariance will be fulfilled, the variables 
in the different batteries must be carefully selected. If one battery 
is built up mainly of very conglomerated variables, e.g. a mixed 
IQ test or sums of school marks, and the other battery of very 
homogeneous factor tests, it is difficult to attain invariance 
between the two batteries. 

The invariance problem is further discussed from a mathemat- 
ical viewpoint in Chapter VII. 

III. Uniqueness.—In developing his factor analytical meth- 
ods, Thurstone started from the principle that an infinite 
number of solutions are applicable to the same factor problem. 
It is therefore necessary to include certain restrictions to make 
the solution unique. 

For the sake of clarity, Thurstone makes a distinction between 
two related kinds of uniqueness. The first is of a mathematical 
Nature and is applicable only to one correlation matrix. This kind 
of uniqueness implies that the limiting requirements should be 
Such, that there is only one matrix А, which fulfils these re- 
quirements and which also satisfies the algebraic relation 


R = Anp Ay. (59) 


The second kind of uniqueness is present if the factors are 
unique in the sense that they represent the only acceptable model 
of the underlying processes studied. As previously mentioned, 
Thurstone adopts a fundamental relativism in regard to this 
question, and believes that there may be several alternative 
models which are equivalent. But, if uniqueness in the above 
sense is desired certain limiting requirements have to be imposed, 
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which result in that only one configuration of factors will occur 
in repeated investigations with different test batteries within the 
same domain (Thurstone, 1947, p. 333). This requirement for 
uniqueness is, as can be seen, closely related to the need for 
invariance. 

The two requirements for uniqueness signify a need of re- 
quirements that at the same time can give an acceptable factor 
model, from a scientific viewpoint, and a mathematically unique 
solution of the relation R = 4,4 An. 

IV. FuxcTIONAL Uxiry.—The concept of functional unity 
is based upon the theory that response patterns to certain types of 
stimuli are more or less closely related to each other or overlap 
in the sense that partly the same processes are involved. This 
grouping of response patterns or their elements may, for ex- 
ample, be the result of the same genetic basis or that the environ- 
ment has had a similar effect on them. Functional unity can, for 
ow be characterized in the following way (Cattell, 1946, 
p. 71): 


dus the unity of a set of parts is established by their moving—i.e., 
appearing, changing, disappearing—together, by their exercising an 
effect together, and by an influence on one being an influence on all. 

It may, for example, be assumed that results obtained in 
different kinds of tests for speed of reaction are dependent on a 
certain center in the central nervous system, which gives rise to 
a high correlation between the results. A numerical factor, for 
example, may partly be the result of training in a number of 
processes at school, which are related and highly dependent on 
each other. Such interrelated response patterns will operate aS 
relatively fixed groups of processes, called by Thurstone, 
“functional groupings” or “functional unities". 

To what extent factors of the “functional groupings” type 
are present, and their degree of stability, is an empirical question. 

V. AGREEMENT WITH NON-FACTORIAL Frnpincs.—Though it 
is important that factors show numerical or configurational in- 
variance between different factor studies, the decisive criterion 
of the value of the factors is, however, the extent to which they 
are fruitful in non-factorial psychological research. 


б 
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According to Thurstone, what is needed is a combination of 
factorial and non-factorial research in the solution of a structure 
problem. In his opinion, the main use of factor analysis is to 
give a first outline of a new domain (op. cit., p. 56): 


The exploratory nature of factor analysis is often not understood. 
Factor analysis has its principal usefulness at the border line of science. 
It is naturally superseded by rational formulations in terms of the science 
involved. Factor analysis is useful, especially in those domains where 
basic and fruitful concepts are essentially lacking and where crucial 
experiments have been difficult to conceive. The new methods have 
a humble role. They enable us to make only the crudest first map of a 
new domain, but if we have scientific intuition and sufficient ingenuity, 
the rough factorial map of a new domain will enable us to proceed 
beyond the exploratory factorial stage to the more direct form of 
psychological experimentation in the laboratory. 


Factor analysis and other methods can be used in combination 
so that factor analysis generates hypotheses that are verified by 
other methods or so that factor analysis is testing hypotheses 
formulated on basis of other methods. 

In order to find factors that meet the above requirements, 
Thurstone uses the principle of simple structure. The methods 
based on this principle are used by Thurstone not only for testing 
hypotheses but also for generating hypotheses. This procedure 
is discussed in Chapter VI. 
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CHAPTER III 


DIFFERENT APPLICATIONS OF FACTOR ANALYSIS 


From the above historical outline it is evident that opinions 
are divided regarding the assistance factor analysis can give in 
solving problems in the behavioral sciences. Here follows an 
attempt at presenting a more systematic discussion of the types of 
questions to which factor analysis can be applied and of the 
methodical problems involved. The major areas are (1) to find 
the number of dimensions; (2) to find descriptive factors; and (3) 
to find explanatory factors. 


The Problem of Finding the Number of Dimensions 


Factor analysis has made important contributions to the be- 
havioral sciences both directly and indirectly by emphasizing and 
defining more strictly the concept of dimensionality. It has be- 
come more evident in many areas that a large number of the 
variables used are multi-dimensional. The concept of dimension- 
ality is not new of course. It is involved, for example, in some of 
the theories, which were advanced before this century, as tO 
colour vision and olfaction. But the models and mathematical 
methods of factor analysis can make these theories more exact 
and help also in testing and refining them. New fields have 
been established to a large extent because of influence of factor 
analysis. Multi-dimensional scaling and latent structure analysis 
are examples of this. 

Here dimensionality problems in the theory of mental test will 
mainly be discussed. For example, in selection and construction 
of a test battery for prediction, one wants to find the smallest 
number of dimensions measured by the tests. Expressed mathe- 
matically, this is the same as finding the rank of the matrix 
comprising the correlations between the tests. 
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In calculating the rank, two problems may be differentiated, 
depending on what type of correlation matrix is used: 

(1) To find the rank of the correlation matrix R, with unities 
(or possibly reliabilities) in the diagonal. This case is connected 
with component analysis. 

(2) To find the rank of the correlation matrix R with commun- 
alities in the diagonal. This case is connected with factor analysis. 

In the first case it is a question of finding the smallest number 
of factors, which can explain the total variances in all tests. When 
the battery contains mental tests, the number of dimensions is 
nearly always equal to the number of tests, since each test 
partly measures something which is specific for it. 

In certain cases, however, the correlation structure may be 
such, that a considerably smaller number of factors than the 
number of tests almost completely explains the variances and 
covariances. Such a case is considered, for example, by Guttman 
(1950) in his scale analysis. By using scale analysis Guttman 
endeavours to test the homogeneity of a scale. The requirement 
for homogeneity, according to Guttman, is, that all items in 
the scale are completely correlated, i.e. that the rank is one of 
the matrix R, with correlations between items and with unities 
in the diagonal. This implies that all correlations are of size one. 
This requirement is never fulfilled in practice. Guttman measures 
homogeneity by the coefficient of a reproducibility, which has its 
maximum value, unity, when the scale is completely homogene- 
ous. In most cases the degree of homogeneity is considered to be 
sufficient when the coefficient exceeds 0.9. Certain assump- 
tions must, however, be fulfilled, e.g. concerning the frequences 
of different answers to the items in the scale. 

The testing of homogeneity by Guttman’s method is a kind of 
component analysis. It is examined if оле component can герго- 
duce practically the total variance in all the different items of the 
scale. In a homogeneous scale, according to Guttman, all items 
must measure the same dimensions and differ in the frequencies 
ОЁ various answers. Guttman's discussion refers mainly to the 
field of attitudes. Similar ideas for the field of power tests have 
been developed by Loevinger (1947). 
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In the second case, a correlation matrix R with communalities 
in the diagonal is considered. The intention here, is to find the 
smallest number of factors that can describe the relations between 
the different tests, i.e. the common parts of the variances in the 
tests. A concept of homogeneity, different from that accepted by 
Guttman and Loevinger, is related to this case. A scale composed 
of items, which correlate one with each other, is very difficult 
to get, especially if the scale should have predictive capacity. 
Another type of scale, more useful for prediction, is a scale, 
composed of items which all measure the same general factor, 
and where each item measures one specific factor. In such a 
case, matrix R, with correlations between items and with com- 
munalities in the diagonal, will have rank one. This concept of 
homogeneity is mentioned, for example, by Cureton (1951, 
p. 646). 

Even in a scale which is composed of such items and which 
measures also specific factors, the general factor will have a 
tendency to dominate the total variance because the general 
factor of each item has correlation one with the same factor in 
the other items. The different specific factors are, on the other 
hand, zero-correlated reciprocally and with the general factor. 

For one individual in a test scale composed of n items, each 
of which is loaded with the general factor and one specific 


factor, the total score P is expressed in the following factor 
equation: 


n 
p Ug 24 Fu Za v Us. (60) 
The total variance of the test scale is 
N n 2 N n n 2 
A9). A (Anut 5ш) 
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| Since all unique factors are uncorrelated both mutually and 
in respect of the common factor, the equation is 
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It is evident that the general factor will contribute to the size of 
the test variance also through the term 


non 
> 2 адад. 


For example, when aj = аз for all items, the general factor 
will contribute z/(z + 1) of the whole test variance, in spite of 
the fact that it only corresponds to half of the variance in each 
item. In such a scale of 24 items, for example, 96 per cent of 
the variance will refer to the general factor. Analogous relations 
can be shown to exist when the general factor has other loading 
values or when several common factors are measured by the test 
scale. This condition does not seem to have been formalized 
in this way before. 

'The rank of the matrix can, of course, be calculated by 
methods other than factor analysis. 'l'he latter procedure, how- 
ever, has often the advantage that the factor loadings found can 
then be used in a subsequent analysis of the data, in order to 
find which the dimensions are. s 

In testing the rank of a correlation matrix, it is important to 
determine the confidence limits for the residual correlations, 
after a certain number of factors are taken out from the correla- 
tion matrix. This problem is treated in Chapter VIII. 


The Problem of Finding Descriptive Factors 


Factor analysis makes it possible to refer the common parts of 
the variances, in a number of observed variables, to a few factors 
whose intercorrelations are either low or zero. The common 
factor loadings reproducing the information in the n(n—1)/2 
intercorrelations of correlation matrix R, are then concentrated 
in the z x m loadings of matrix An. There is an unlimited num- 
ber of matrices of rank m, which are all alternative and inter- 
transformable descriptions of R. Consequently, it is a question 
of selecting a solution with the most suitable characteristics. 
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More or less definite hypotheses about the nature of the 
descriptive model are often constructed. Factor analysis can 
then be used for testing whether a matrix A,,, which corre- 
sponds to these hypotheses, can be fitted to the correlation matrix 
R. If this is the case it proves that the hypothetical model is an 
acceptable description. Other considerations and investigations 
—whether factor analytical or not—will then determine if this 
or some other factor matrix is the best description for a certain 
purpose. 

In many cases there is no particularly specified hypothesis 
available that implicates a certain structure of the matrix Am. 
'The structure has to be selected according to more general 
scientific principles. These may, for example, be obtained from 
the special field of research, from statistical theory, or from ge- 
neral scientific principles. A principle often used is to reduce 
the number of factors m in the matrix 4, to the least possible 
number, that is the rank > in the correlation matrix, R. An 
additional method for getting a simple description is to use or- 
thogonal factors, i.e. a solution A,, chosen in such a way that the 
matrix ¢, including the correlations between the factors, is a 
unit matrix. 

Sometimes it may be better to maximize, by a principal 
component solution, that part of the common factor variance of 
the battery, which is taken out with the first factor, and then 
apply the same for the second factor, and so on. In this case, 
it may happen that a number of factors, fewer than rank 7, 
reproduce such a great part of the correlation matrix that the 
residuals can be neglected, when this smaller number of factors 
has been taken out. 

A principle common to several methods, e.g. the simple struc- 
ture method and the bi-factor method, is that they shall fulfil the 
requirement for a certain number of zero-loadings in the matrix 
Am. Each of these methods has, as shown in Chapter II, special 
requirements in regard to the distribution of the zero-loadings. 

The above-mentioned and other principles are employed in 
different combinations in the various factor analytical methods. 
In certain cases, different principles conflict with each other, and 


- 
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it is necessary to decide which one should be given priority. In 
the method of simple structure, for example, the requirement for 
a certain distribution of zero-loadings is given priority over the 
orthogonality, while for the bi-factor solution, it is rather the 
opposite. 

It is not the intention here to go into a detailed discussion or 
to give a systematic account of the various alternative methods. 
This is done, for example, by Holzinger-Harman (1941) and 
Cattell (1952). It is sufficient to point out that it is a question 
of a conventional choice, where criteria such as the special aim 
of the description, its simplicity, and statistical efficiency, are 


the deciding principles. 


Descriptive Factor Analysis and Its Applications 


In everyday life and partly also in scientific psychology, a 
very extensive vocabulary referring to a lot of abilities, person- 
ality traits, symptoms, etc. is used to describe human behaviour. 
An example of this is the list of about 18,000 English terms 
designating behavior compiled by Allport and Odbert (1936). 
Many of these words are synonyms but others are intended 
to denote some shade of meaning associated with a particular 
word. A description of a human being in a number of categories 
is, of course, too comprehensive. The list must be reduced to 
a few essential variables. In other words, it is a question of 
selecting, from the universe of all possible variables, a suitable 
group which makes possible a rather complete but still simple 
description. { Т 

Factor analysis is a useful tool for such a reduction, as it gives 
a description of a large number of variables, in terms of a few 
relatively independent factors. : 

Fanor апага] research programs, of the type outlined 
above, have been carried out, for example, by Burt, Holzinger, 
and Cattell. The factors found in these studies may be regarded 
as being relatively fundamental, descriptive papel Hoan 
they can be someting more, is a question to be treated later in 


this monograph. 
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Factor analysis can be carried out either as an exploration or 
as a test of hypothesis. In exploratory studies, the factor analysis 
is carried out principally without definite hypotheses as to the 
occurrence of a certain factor structure. The selection of a 
particular factor analytical method involves, however, a choice 
of the general type of the factor analytical model. In a solution 
based on, for example, a general factor, a large part of the total 
variance in the battery is compressed into a general factor, while 
in a solution applying the principles of simple structure, a gen- 
eral factor of the first order is avoided. 

In hypothesis-testing, factor analysis is made according to 
theories established in advance that certain tests are loaded with 
certain definite factors. Factor analysis is carried out in such a 
way, that these factors, if present, will be apparent. For example, 
if one wants to test the hypothesis that results in geometry can 
be described in terms of a general ability factor, in addition to 
a spatial factor, then a test battery, including ability tests of 
different types, also spatial, and various tests of achievement in 
geometry, are selected. The battery is administered to a sample, 
where individual differences both in general ability and in 
geometrical aptitude are expected to be present. The resulting 
correlations are factor analyzed, e.g. by Holzinger's bi-factor 
method, according to the hypothesis that a general factor is 
present which is loaded also in the geometry tests. If the hypo- 
thesis is correct, a general factor will be obtained which is loaded 
in all tests, plus a group factor having loadings in the spatial and 
geometrical tests, but not in other tests, for example, the verbal 
tests. If this factor model is applicable to the data it shows that 
in such a population geometrical results can be partly described 
in terms of a general factor and a spatial factor. On the other hand, 
it does not signify that this model is either the most suitable, 
or the only possible one, since there are other factor analytical 
models that give alternative descriptions. Other criteria, for 
example the usefulness of the model in test construction and in 
educational connections, must be used to decide whether the 
model is the most suitable description of the domain. 

The main advantage of factor analysis, as an aid in finding 
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descriptive concepts, lies in the fact that it can help in finding 
factors that are easier to comprehend than the multiplicity of 
tests. Instead of carrying out more detailed observations and 
experiments on a single test, in an attempt to study what it 
measures, the correlations between a large number of tests may 
be used for this purpose. The intercorrelation matrix indicates 
to what extent the different tests measure the same thing, and 
this in itself is an important basis for drawing conclusions about 
the nature of the characteristics or factors of which the tests are 
a measure. Factor analysis of the intercorrelation matrix gives still 
greater possibilities of comprehending and investigating the nature 
of the factors measured by the tests, since the factor matrix 
summarizes, in a few factors, the information included in the 
intercorrelation matrix. From the factor matrix, it is evident in 
which test each factor has high and low loadings respectively, 
and in which tests loadings are lacking. This gives a broad basis 
for a psychological analysis whose purpose is to identify factors 
that describe, in a simple way, the connection between the tests. 

Finally, it may be stated that through factor analysis one can 
get suggestions for descriptive models that include a few rela- 
tively independent, and objectively established, parameters. Factor 
analysis can also be used for the strict testing of hypotheses 


about suitable descriptive models. | 
Some examples of the application of factor analysis for de- 


Scriptive purposes will now be given. ms Y. 
A promising field of application for factor analysis is studies 
of changes in the content of the variance of a certain test, when 
the individuals are subject to different types of influences, such 
as growth and practice. The same technique applies when a 
test is given under different testing conditions, for example, after 
changes in time limits. А ў 
Investigations of this kind may be made in the following way. 
he test is given to a sample of individuals under different condi- 
tions, for example during different phases of a learning process. 
The results obtained on the different occasions form one variable 


for each occasion. A battery of carefully selected reference rt 
measuring well known factors is given to the same sample. 
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PERCENT 


VARIANCE UNACCOUNTED FOR 


NUMERICAL FACILIT 


PERCEPTUAL 
SPEED 


COMPLEX COORDINATION, SPECIFIC 


STAGES OF PRACTICE ON THE COMPLEX COORDINATION TEST | 


Fig. 3. Percentage of variance (shaded area) represented by each factor at 
different stages of practice on the Complex Coordination "Test. (From Fleish- | 
man & Hempel, 1954.) 
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These factors are expected to be loaded in the variables formed 
by the test studied under different conditions. Correlations be- 
tween all the variables are calculated and subjected to factor 
analysis. The factor solution is rotated according to the factors 
Which the reference tests are expected to measure. By this 
operation, the variances of the variables, based upon the different 
testing occasions, will be described in terms of the known 
factors which are measured by the reference tests and by other 
factors common mainly to the variables based on different testing 
occasions. In a factor matrix containing such factors it is possible 
to get a clear and useful picture of the changes in factor structure 
from one occasion to the other. 

А good example of this approach is Fleishman and Hempel's 
(1954) study of the effect of training on performance in a com- 
plex psychomotor test. It was found that the longer the training 
continued, the less extensive became the proportion of the 
Variance in the complex psychomotor test, which corresponds to 
factors measured by the reference tests. This is shown in Fig. 3; 
'The special “psychomotor” factors became more dominant, 
the longer the training continued. Results of this type have 
implication both for learning theory and for prediction of 
abilities that are to be trained. " . 

Another field of application is item-analysis, in connection 
With the construction of scales, questionnaires, etc. Earlier it 
has been discussed how factor analysis may be used in the con- 
struction of homogeneous scales. The method can also be used 
in studies of which dimensions are involved, when the matrix R 
Of correlations between items has a high rank. In such factor 
analyses many special problems occur that are not yet M ed 
Solved. They relate, for example, to methods for calculating 
correlations when the levels of difficulty vary between the items 


(Vernon, 1950 b and Gourlay, 1951). 
Prediction and Factor Analysis 


Prediction and the requirements implied by this in "EE ie 
the variables, are important psychometrical problems that have 
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been rather thoroughly dealt with from a theoretical viewpoint. 
A systematic discussion of this topic is found, for example, in 
The Prediction of Personal Adjustment (Horst et al., 1941), where 
the role of factor analysis is mentioned, and also in Theory of 
Mental Tests (Gulliksen, 1950). 

The selection of tests for prediction gives illustrative examples 
of the problems that arise when looking for suitable descriptive 
variables, and of the aid factor analysis can render. There are 
two types of problems of prediction to be discussed in this 
connection: direct prediction and differential prediction. The 
purpose of direct prediction is to predict, with the greatest pos- 
sible validity, the subject’s results in a certain field. The selection 
of personnel for a certain job, or of pupils for a certain type of 
training are examples of this. Often the same set of predictors 
should be used for direct prediction in many fields at the same 
time; this is the problem which will be discussed here. For this 
purpose, a number of tests! are needed, which are chosen in 
such a way that they give, in a simple manner the best possible 
predictions of results in all the jobs mentioned. From a psycho- 
metric viewpoint, the problem may be said to consist in finding 
a limited number of tests that, weighted together with suitable 
weights, give maximum correlations with criterions of success 
in respective jobs. The problem of differential prediction arises 
when a limited number of people are to be given different jobs 
according to assigned quotas, so that the total sum of job results 


is maximized. This problem will be considered later. 


The methods for planning, construction and selection of a 


test battery to be used for direct prediction of success in a number 
of jobs simultaneously, are relatively well developed. Therefore, 
it is sufficient, to mention the different steps briefly and con- 
centrate the discussion on the contributions which can be made 
by factor analysis. 

(1) Јов CLASSIFICATION. 


i — The aim of this procedure is to 
find the jobs that are si 


milar in important and measureable 


1 The argument is, in principle, appli 
ES p » applicable even to other types of observed 
nm for example biographic and physical data, but is here applied to 
tests оп у. Further, the assumption is made that all tests are equal from ad- 
ministrative points of view such as cost and testing time. 


^ 
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respects and can be grouped together in so-called job families, 
in which approximately the same traits are required for success. 
Factor analysis can assist in making such a grouping simpler and 
more objective. Coombs & Satter (1949) have made such a 
factor analysis of job descriptions using correlations based on 
the number of common elements in the descriptions of each job. 
The dilemma in grouping jobs in job families is, that if many 
jobs are assigned to one family, what is typical of the different 
jobs will, to a certain degree, be neglected in favour of more 
general properties. If, on the contrary, one goes to the opposite 
extreme and keeps the special jobs almost completely separate, 
the number of job families will become too large to handle 
conveniently and at the same time many of the properties will be 
very specific. Hence some kind of compromise between the two 
extremes must be made. Grouping in job families cannot be kept 
separate from the job analysis, which is the next step. 

(2) Јов AwaLvsis.— This step aims at analyzing job results 
into, if possible, measurable categories, and at determining 
how important each one of them is for success in the job. A 
number of different methods may be employed for job analysis 
(Kerschner, 1955). Factor analysis has also been used in this 
connection, primarily for systemizing and limiting the number 
of factors in the job descriptions. McQuitty et al. (1954) have, 
for example, factor analyzed about 150 descriptive phrases that 
have proved to differentiate between good and poor flight me- 
Chanics, and have reduced these phrases to 23 factors. 

(3) Tzsr CowsrRUcTION.— The success of this is, of course, 
to a great extent, dependent on the quality of the job analysis. 
The selection and construction of the tests can be made primarily 
along two lines, namely, in the form of analogous tests or of 
analytical tests or of some compromise between these two types. 
Analogous tests are constructed, in such a way as to present the 
testees with tasks that are similar in a limited form, to the tasks 
that are later to be carried out in the job, i.e., they are analogous 
to the job requirements. Ап example of this isa i poni 
learning test in which the subjects, after having learned some 
simple Morse-letters, are tested in their ability to receive these 
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letters. An analogous test is often an effective predictor for exactly 
the job it is intended for, but generally has a poor predictive 
capacity for other jobs because of its specific character. In the 
construction of a test battery for the prediction of a whole series 
of different jobs, it will often be necessary to have at least one 
analogous test for each job. 

The analytical tests are intended to measure the important 
elements of the jobs in question. From a practical viewpoint, 
each of the tests ought to measure an element which is so funda- 
mental that it can be used for several jobs. Tests measuring such 
fundamental elements will have a wider range and will, on the 
whole, be more applicable and flexible than analogous tests. The 
main principle for construction of analytical tests must be to 
find, by building on the job analysis, elements that have validity 
for many job families, and then to construct tests for the most 
important of these elements. By employing analytical tests it is 
possible to make predictions for a number of jobs, by means of 
a few tests. For this to be successful, it is necessary for the differ- 
ent tests to have both high validity for many job criteria, and 
low intercorrelations. The low correlations between the tests 
Show that they measure different elements, i.e., cover different 
parts of the variances in the job results. 

The importance of this is clearly evident if one studies which 
correlations a new test, called л + 1, should have, if it is added 
to a previously used battery of л tests, in order to increase its 
predictive power. 


The calculation is based on the following correlation matrix: 


1 Toag...n) To(n 1) 
Р, = | т 
1 002... n) 1 Ta2 ...2) (ni (63) 
Тот +1) Та? „пуз 1 


In this case 79(5...,) is the multiple correlation between the 
criterion and the original tests, ғ, (44) Is the correlation between 
the criterion and the new test, and Ta2---5) (44) 18 the correlation 
between the original battery and the new test. The original 
battery is weighted together with the same weights as against 
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the criterion. The squared multiple correlation between the 
criterion and the test battery including the new test is as follows: 


2 
TAZ... n+) 


2 2 
Tõaz...n T Tóm) — 27002... Ton a2...) 021 © (64) 


3 
1— Ta2...n)(n+1) 


By means of this formula the following conditions are found for 
the increase of the multiple correlation: 


If оу 27 (3---5 then the multiple correlation will always 


increase. 

If ong) = Toners then the multiple correlation increases 
when fassen (41) € 1- 

If 795,44; = Rroas- «2». then 7з... пул) Must be less than А 


if the multiple correlation is to increase (k<1). 


Generally speaking, if a new test is able to increase the mul- 
tiple correlation, then the quotient of the correlation between 
the criterion and the new test, and the correlation between the 
original battery and the new test, must be greater than the corre- 
lation between the criterion and the original battery, i.e. 

(65) 


To(n41) >Toa2 a 


Ta...) (41) 


A new test can improve prediction through its high validity or 


through its low correlation with other tests or through a com- 


bination of both. 

Factor analysis is a useful tool for the development of tests 
With the above-mentioned characteristics. The construction. of 
а test battery for the purpose of prediction, consists, according 
to factor analytical terminology, in finding the factor structure 
of the job results and in designing tests that are as pure measures 
Of these factors as possible. | 

The role of job analysis as a basis for test construction, and 
how factor analysis can facilitate such job analyses, has previ- 
ously been discussed. In order to make job analysis useful, its 
Tesults must be presented in a terminology that fits the frame of 
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reference in which test results are described. The discrepancies 
in the terminologies of job analysis and of test construction are 
unfortunately great. This difficulty can be avoided by factor 
analyzing criteria of success in respective jobs, together with a 
test battery composed of reference tests with a factor structure 
well established through earlier factor analyses. Then the cri- 
terion variance will, to the extent that the tests are correlated 
with the criterion, be described in terms of a number of relatively 
well-defined and well-known factors that have low intercorrela- 
tions. In this case, a whole test battery makes it possible to obtain 
a better survey of the criterion structure, and a great many 
hypotheses as to factors of importance for success in the job, can 
be tested simultaneously. On the basis of the first results, new 
hypotheses as to the factor structure of the criterion can be 
advanced; these hypotheses lead to the addition of new or 
reconstructed tests in the next analysis, and so on, until as 
much as possible of the criterion variance is covered by well- 
known factors. Such methods have been used, for example, in the 
U.S, Air Force; and have been further discussed by Guilford 
(1948). 

The selection of tests to be included in an analysis of the 
above type is now simplified through the monograph The 
Description of Aptitude and Achievement Tests in Terms of Rotated 
Factors by French, (1951). It contains descriptions of well estab- 
lished factors and of tests that measure them as well as possible. 
The purpose of that monograph and a rather optimistic opinion 


on the capacity of factor tests in prediction, are stated in its 
introduction (p. v): 


'This book is devoted to the Progress of test development toward 
the situation where the test constructor has a file of tests to measure 
each factor of the mind. Within each file will be hard tests, easy tests, long 
tests, and short tests, but they will all measure in pure form the factor 
named on the label of the file. When the test constructor of the future 
gets an order for a tailormade test, he will consult his summaries of past 
researches and determine what factors need to be measured and what 
weights should be used with them in arriving at a test battery having 
maximum efficiency. Should the problem posed by his client be of so 
novel a nature that no past researches apply, it will be possible simplY 
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to turn to a file labelled “Ready-to-use Diagnostic Batteries" and select 
a suitable collection of standard pure-factor tests which reveal the facto- 
rial composition of a suitable criterion population. 


On the basis of his studies of earlier analyses and his own 
investigations French has also published a series of tests that 
measure the various factors as well as possible (French, 1954). 

Often there are no criteria available in due time or they are so 
defective that the preliminary stages of the selection of the test 
battery must be made without criteria. The selection of tests 
must then be based primarily on “construct validation" (Cron- 
bach&Meehl, 1955). In this procedure, statistical comparisons 
with criteria of success are replaced by psychological considera- 
tions and indirect proofs, which indicate that the tests measure 
what they are intended to measure. Factor analysis facilitates 
such determination of the test structures, especially if the pro- 
posed tests are included in a battery of reference tests with a 
well-known structure. One can then get, in the factor matrix, 
safer and more objective grounds for the identification of what 
the tests measure. 

(4) Tur Рдовгвм or Wrronrs.— Гће last stage in the develop- 
ment of a test battery for prediction is, the determination of the 
Weights the test should have in prediction for the various jobs. 
A method which is theoretically correct, is to determine the 
multiple correlation between the job criterion in question, and the 
test battery, and to take the regression coefficients obtained as 
Weights. Factor analysis is not a substitute for this procedure, 
since factor scores are only weighted sums of test scores, and 
cannot increase the correlation that is already maximized through 
the weights obtained in the calculation of multiple correlation 
(Thomson, 1948, pp. 114-116). In order to work well, the m 
tiple correlation procedure requires, however, an unbiased an 
relevant criterion. If this is not the case, it becomes more im- 
Portant to base the weights on knowledge of the job s 
Tents and according to the nature of the tests that can be ob- 
tained from job analyses, factor analyses, etc. : 

The determination of regression coefficients by the calculation 
of multiple correlations is very sensitive for sampling errors. 


5 — 576055 S. Henrysson 
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This has resulted in an emphasis on the requirement for the 
cross validation of the tests on new samples (Mosier, 1951 a). 
The fluctuations in the regression coefficients and the drop in 
the value of the multiple correlation after cross-validation become 
greater as more tests are included in the battery, and as the sam- 
ples get smaller. These difficulties have been discussed in detail, 
for example, by Horst (1941), who points out the importance 
of reducing beforehand the number of tests that are to be in- 
cluded in the regression equation. He also mentions that factor 
analysis is an effective aid in this respect. On the basis of factor 
analysis, a smaller number of pure-factor tests may be substituted 
for a larger test battery of complex tests. 

Efforts have also been made to use factor analysis as a step in 
regression analysis itself. The aspects of factor analysis, which 
are found attractive, are the possibilities of condensing the 
common variances in the correlation matrix R to a matrix Am 
including only a few orthogonal factors. Especially when R is 
to be used in many mathematical calculations, for example, in 
the estimation of multiple correlations and regression weights, it 
might prove advantageous to adopt what is often the harder 
method, and first estimate A,, (Dwyer, 1940, and Guttman, 
1940). 

One „use of the common factor matrix, when calculating 
regression equations, is for finding a system of weights which in 
cross-validation gives rise to less shrinkage of the multiple correla- 
tion. Leiman (1952) has tried to use the common factor matrix 


in the calculation of regression weights; and this is his reason for 
doing so: 


, "The rationale for this is that, since the number of factors is ordinarily 
ess than the number of independent variables in the system, fewer 


degrees of freedom will be lost, and if № remains the same, the standard 
errors of the regression weights will be smaller. 


Leiman tested this theory on a population of 1,468. In the 
first study he divided the Subjects into samples of 30, and in 
the second study into groups of 200. The result was a significantly 
higher cross-validity for the common factor regression weights in 
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the first case, but they were not significantly higher in the second 
case, when compared with regression weights calculated in the 
conventional manner. 

A drawback, if the identical results of calculations are required 
Which would be obtained by conventional regression analysis, 
is that, in practice, the factor matrix A,, never fully exhausts R. 
There is always a certain residual variance left out, even if 
it is kept within the limits of sampling errors by the use of a 
significance test. Another reason against the use of factor analysis 
in this connection is the rapid development of electronic comput- 
ers. This will mean that labour and time elements in numerical 
calculation is less important. 

As has already been mentioned, in regression analysis a reduc- 
tion in the number of tests is required, so as to obtain more stable 
regression weights after cross-validation. Various methods have 
been discussed for excluding such tests that either do not increase 
at all, or increase the size of the multiple correlation only very 
slightly, as compared with the size it acquires by means of a few of 
the tests. Dwyer (1940), among others, took up this problem and 
mentioned that factor analysis can assist in reducing the number of 
tests. This idea was still further developed by Lubin & Summer- 
field (1951). They used the diagonal method of factoring for 
finding those tests, which taken together, are the best predictors. 
They take as their starting point the test that has the highest 
correlation with the criterion, and then find a second test which, 
when added to the first, will, to the greatest extent, increase the 
predictive ability, and so on. Finally the tests will be left which 
can only increase the predictive ability of the battery to a very 
small extent and which may be excluded. . 

A rougher method, which is often better, however, jf the 
criterion is poor or complex, is to weight together as to com- 
Posites”, those tests which measure the same common ad 
These composites may be relatively few and low-correlated an 
can be more easily weighted together in a regression equation 
to stable total scores. This grouping of te 
often be done by inspecting the correlation m 
ularly if the correlation structure is complex, 


f tests into composites can 
atrix; but, partic- 
the factor matrix 
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can produce a simpler and clearer picture of the relations between 
the tests. 

Another possibility suggested by Dwyer is to factor analyze 
the battery, including the criterion, and then to extract one test 
for each factor that has loadings in the criterion. This, however, 
presupposes that it is possible to find relatively pure measures of 
each factor. If this is the case, the multiple correlation between 
the criterion and the m number of factor tests should be rather 
close to the multiple correlation given by the whole battery of л 
tests, and in addition should possess the advantage of being 
considerably more stable in cross-validation (Dwyer, 1939). 

Both the method of using composite scores and that of single 
factor tests require that the tests are rather pure measures of the 
factors included in the criterion. In practice, it is often very 
difficult to fulfil this requirement adequately, in any case in 
regard to paper and pencil tests. 


Differential Prediction and Factor Analysis 


In the previous section are considered the requirements which 
a test battery has to satisfy in order to afford direct prediction 
in respect to each of a number of jobs. The object is to obtain 
the highest possible multiple correlation between a battery of 
few tests and each of the criteria of job success. In most cases 
the various job criteria are more or less intercorrelated. Conse- 
quently even if relatively uncorrelated factor tests are employed, 
these will, with weighting systems, which are largely similar, be 
present in regression equations for the prediction of several 
jobs or job families. Another result of the high correlations 
between the criteria, is the well-known fact, that general measures 
of ability, for example, a general intelligence test, often give rather 
good prediction for a number of different jobs. If such a test 
dominates the selection for all these jobs, then practically the 


same individuals will be considered the most suitable for all the 
jobs. This is not a draw! 


back from the viewpoint of a pure per- 
sonnel selection proced 


QNEM lure, when the number of applicants is 
large—in principle, unlimited —in relation to the demand. On 
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the contrary, selection by such generally useful tests can often 
save time and effort, since prediction to all the different jobs can 
be made by only using a few tests. 

The problem is different if the number of applicants is limited, 
and if most of the applicants must be placed in some of the jobs 
the selection procedure comprises. This is the actual position, 
for example, in the military classification process, where each 
conscript who fulfils certain minimum requirements of a psy- 
chological and physical nature must be placed in some job ac- 
cording to the personnel quotas established. The classification has 
to be made in such a manner, that the total result, according 
to the existing productivity criteria, is maximized. 

How to maximize the result in accordance with the existing 
knowledge of the probable level of success of each individual 
for each possible assignment, is at present under investigation. 
A number of different attempts have been made to solve this 
problem (Votaw & Dailey, 1952, and Dwyer, 1954). The main 
Principle is indicated in the following quotation from a report by 
Votaw & Dailey (1952, p. 1): 


The purpose of the research summarized in this bulletin is to develop 
techniques of assigning the available personnel to the available jobs in 
such a Way as to maximize the efficiency of utilization of manpower. 
An important assumption regarding the techniques is that the assigning 
agency can determine a quantitative index for each possible person-job 
Pairing, representing the person's productivity or pay-off to the service 
if assigned to that particular job. The manpower will be utilized with 
maximum efficiency if the assignment of all persons to all jobs is such as 
to maximize the sum of the pay-off amounts for the finally determined 


combination of persons and jobs. 


If personnel classification by such a method is to be successful, 
it is not enough for the tests to give good prediction only for T 
Опе of the various jobs. As previously mentioned, this айр ten 
be done just as well by certain general ability tests, that "skim 
Off the cream". An example is a general classification test of the 
kind used by the Armed Forces in many countries. Such a test 
has proved to give good prediction also for most jobs in the 
Swedish Armed Forces. A personnel classification system which 
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is mainly based on such a test leads to a great demand for per- 
sonnel with high scores in just this test, and the supply of highly 
qualified personnel to be drafted in this way will be insufficient. 
The consequences of this shortage are beginning to be taken into 
consideration, particularly in the United States, by both military 
and civilian organizations in their search for high level talent. 
'The problem has been discussed from a psychometric view- 
point for example, by Daily (1951; pp. 17-18) in an article 
entitled The Supply and Identification of High Level Talent, 
where he writes: 


'The supply of high level talent will be in part a function of the 
number of relatively independent dimensions of individual differences 
that are generally accepted as significant talents. ... We can see that 
if talent is regarded as unidimensional (general intelligence for example) 
the supply of high level talent will be X percent. However, if talent 
is regarded as existing in two independent dimensions, then the supply 
of high level talent becomes X percent plus X (100—X)/100 percent or 
very nearly 2 X percent if X is small as would usually be true. Of course, 
the increase becomes considerably less if the two dimensions are cor- 
related. The supply will be asymptotic to 100 percent as the number 
of independent dimensions becomes large, and even with moderately 


correlated dimensions the supply can become relatively large with a 
reasonably small number of dimensions. 


This type of personnel classification, known as differential 
prediction, must, to a greater extent than direct prediction, be 
based on abilities unique for each job or job family. The more 
the selection for each of the jobs can be based on one special 
combination of tests which is low-correlated with test-combina- 
tions for other jobs the greater is the probability that the different 
jobs will have their personnel quotas filled without competition 
for the same type of personnel. This presupposes that the tests 
are valid only for certain occupations and are mutually low- 
correlated. This is emphasized, for example, by the experts of 
the U.S. Air Force in this field, who have been very active in the 
development of the theory and Practice of differential prediction. 
In a report issued by the U.S. Air Force on “Validity of the 


Airman Classification Battery AC-1” it is stated: (Grage-Gordon, 
1950, p. xix): и 


ka 
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| The ACB was developed upon the rationale of differential classifica- 

tion, which implies that each job or group of jobs requires a different 
pattern of aptitudes. The prediction of success in each of these jobs or 
groups of jobs requires the use of many tests which can be combined 
into weighted composites, depending upon their intercorrelations 
and correlations with the criteria of success on the job. The most effi- 
cient classification occurs when the separate tests, each as pure as pos- 
sible, measure different aptitudes, or correlate low with one another and 
high with the criterion. The differentiation possible by use of such 
weighted composites depends on their validity for differential prediction, 
which, as is indicated above, is a function of differences in validity 
coefficients and the size of intercorrelations of tests. 


An essential requirement for differential prediction is low 
intercorrelation between tests, since the prediction for each 
individual is also based on differences in aptitudes for the different 
jobs described, for example, in a profile. This has been shown, 
by Mollenkopf, who gives a formula for the validity coefficient in 
prediction of the difference between two criterion scores a and 
b with the help of the same test battery (Mollenkopf, 1950, p. 
414): 

2 2 „+ Ria —2Rava Roso Raros, 
Riva EUN (66) 


The notation * indicates predicted number. Rye а is the correlation 
between the predicted difference and the actual difference; Rasa 
is the multiple correlation between the battery and the criterion 
а; Ry», is the multiple correlation of the battery with the criterion 
b; rua. is the correlation between the weighted test scores for 
Prediction of respective criteria and ra, is the correlation between 
the criteria а and b. 
Mollenkopf points out that fas x» i.e. the correlation between the 
weighted test scores, is the determining factor for the success of 
the differential prediction (op. cit., p. 415). This is clearly seen 
when R,., = Rye», which gives 
2 „ (re), 
Rise Ris es (67) 


t that Rye, approaches unity 


From this formula it is eviden s 
which is the correlation be- 


When r,.,, approaches Zero. aeoe» 
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tween the sums of weighted test scores, can also be calculated 
as a weighted sum of correlations between tests. The correlation 
T,,. becomes smaller, the lower the correlations are between 
the tests, and the greater the differences in weights for the two 
criteria. 

"There are several discussions on the requirements of differen- 
tial prediction in respect of the tests (Mosier, 1951 b, and French, 
1955). Consequently it is not necessary here to go more fully 
into this problem. It has already been shown that in order for 
differential prediction to be successful, tests are required which 
are valid only for certain jobs and which have mutually low 
correlations. Factor tests are supposed to have at least the second 
property. 

The procedure employed in the construction of factor tests 
for differential prediction has much in common with that which 
has been discussed in connection with direct prediction. A com- 
plication may, however, arise. If all criteria are highly correlated 
and thus have a similar factor structure, they will be correlated 
with the same tests, and the possibility of making differential 
predictions of these criteria will consequently be small. The 
correlations between the criteria can be high, either because the 
jobs that they correspond to are actually highly correlated or 
because the criteria are biased in the same direction. Both these 
cases are present, for example, in the military personnel classi- 
fication procedure. In this case, the criteria are often marks based 
on results of training, which may be expected to be highly cor- 
related. Moreover, there is a tendency to give marks which are 
based perhaps too much on—for all branches of training—com- 
mon variables such as general educational ability, adjustment to 
other conditions in military life, military conduct, and willingness- 

Should the criteria be unbiased but still mutually highly 
correlated, this indicates that the real basis for successful differen- 
tial prediction is small. Performances within the group of occupa- 
tions or job families are in regard to prediction, in other words; 
dependent on the same factors. 

Ап intermediary form between direct and differential predic- 
tion is comparative prediction. It is applicable, for example, to 


i 
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vocational guidance where the client wants to know both which 
D that are most suitable for him (differential prediction) and 
ow well he is suited for them (direct prediction). He needs a 
vocational profile in which the differences between the profile 
values are valid and where each profile value gives valid predic- 
tion of the job in question. French (1955, p. 27) expresses this as 
follows: 
Where there is needed a decision as to which of two occupations or 
ua of study offer an individual the greater chance of success, 
ifferential prediction provides the direct answer. However, in the usual 
case, there is not only interest in which course of action is best but also 
there is interest in how successful each course of action is likely to be. 


. The requirements of the test are substantially the same, both 
in comparative and differential prediction. The validity of each 
test must be limited to certain jobs and the correlations between 
the tests must be low. The factor test is thus also a possible aid 
in the solution of the problem of comparative prediction. 


Conclusions about Factor Tests as Predictors 


Even if it can be maintained that tests based on factor analysis 
Constitute theoretically a rational solution of many prediction 
problems, the practical results hitherto obtained in the form of 
Statistically reported validities are not particularly numerous 
or convincing. There are several reasons for this, besides those 


mentioned above. 


The degree of resemblance be 
families may be such, that it can be described, to a great extent, 


by a general factor. A test measuring this general factor gives 
direct prediction for all jobs, to a relatively high degree, but 
cannot lead to successful differential prediction, which is of 
Particular value. It is necessary to have a test measuring the 


specific variance for each job. І 

Another possibility is, that the requirements for some jobs are 
So specific in their structure that no common factor of importance 
can be found. In this case, tests measuring special factors are 


required both for direct and differential prediction. 


tween the different jobs or job 
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Another difficulty is, that it is not always possible to divide the 
variance in a certain job performance into factors. This is pointed 
out, for example, by Fleishman, who is engaged in constructing 
valid, and at the same time, low-correlated psychomotor tests, 
notably for differential prediction in the U.S. Air Force. He 
says (Fleishman, 1953, p. 45): 


There is an assumption here that it is possible to break down the 
variance in performance of complex psychomotor tasks into simpler, 
more fundamental psychomotor functions. Whether the introduction 
of complexity of function (e.g. tasks requiring coordination or integra- 
tion of operations) introduces variance not reproducible by any number 
of more analytical tasks remains a problem for future research. 


In many cases the requirements for the different jobs can in 
themselves be favourable to both direct and differential prediction 
with the help of factor tests but those criteria for success in the 
jobs, that can be obtained, are so imperfect, that a rational test 
construction program with the help of statistical technique can- 
not be carried out. The criteria have often so low a validity and 
reliability that the unique parts of the variances dominate. Va- 
lidation is also impeded if the criteria are biased in the same 
direction, for example, through the influence of a halo effect. 
It can cause the appearance of an illegitimate common factor that 
complicates validation, particularly of tests for differential predic- 
tion, 

In many cases the criterion of success in a job is complex and 
composed of a number of subcriteria, which should be given 
weights in relation to each other. The technique of canonical 
correlation (Hotelling, 1935) can be used for finding the set of 
weights for subcriteria and tests, which gives the maximum 
correlation between the two groups of variables. Mostly it is, 
however, more relevant to give weights to the different subcriteria 
on non-statistical grounds, e.g. educational or psychological 
considerations. Peel (1948) has dealt with this problem, and has 
developed a method for maximizing the correlation between à 
set of Subcriteria, given certain arbitrary weights, and a set of 
tests. The problem of finding the arbitrary weights for the sub- 


d. More attention has recently been 
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paid to this problem (Ghisselli, 1956 and Sluckin 1956). A 
conceivable way of finding the dimensionality of a criterion 
would be to factor analyse the different parts of the criterion, 
with or without the use of reference tests. This should give a 
picture of the criterion structure and should also contribute to a 
reduction of the number of subcriteria to a smaller set. 
А Even if the criteria are unbiased and of high validity and аге 
m other respect such that the possibilities for differential predic- 
tion are good, the technical difficulties to produce differential 
prediction are very great. Generally, each individual tested, will 
get criterion score only in the job for which he has been selected. 
The theory of differential prediction—for example, formula(66)— 
presupposes, however, that each individual has scores in all 
criteria, The theory works with a matrix including all correlations 
between the different tests and criteria for the whole sample of 
individuals who are tested with the battery. In practice it is not 
generally possible to get correlations between the different criteria. 
Furthermore, the correlations between the tests and criteria will 
be based only on the sub-sample selected for each particular job. 

The statistical treatment can also be complicated by restriction 
of range, caused by the fact that a test which is to be validated, 
has also been used in the selection for the job. If this restriction 
of range is of the uncontrolled type based on other variables, 
then the common correction formulae are not directly applicable 
(Gulliksen, 1950, p. 162). 

It is one thing to establish the 


rion, another to work out the tests t h 
à pure form. Factor tests, particularly those which are useful 


from the viewpoint of time and cost, tend to be impure and 
have a much higher correlation with one another than corre- 
Sponding factors. The tests of Primary Mental Abilities, con- 
structed by Thurstone on the basis of his factor analytical in- 
Vestigations, for example, have these shortcomings. This is 
evident from correlations between the tests of Primary Mental 
Abilities published, for example, by Bond & Clymer (1955). 
The result of these and other difficulties is, that, at present, 
there are rather few consistent examples of how factor tests have 


factor structure for a job crite- 
hat measure these factors in 
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been used successfully in connection with prediction, especially 
differential prediction. Particularly in the United States, however, 
relative optimism is felt concerning the possibilities of achieving 
this. An example of this is the quotation from French on page 64. 
In other countries, especially in England, a more critical attitude 
is often adopted (Vernon, 1950a, p. 128). This is also the impres- 
sion one gets from available investigations from different count- 
ries. At present, it seems as if factor analysis has mainly contrib- 
uted to the development of the theory of differential prediction. 
Practical results are, to a great extent, lacking (Terman, 1954, pp. 
225-226, and Cronbach, 1956, p. 178). 


The Problem of Finding Explanatory Factors. 


A factor analysis of a set of data may be regarded as a deduc- 
tive procedure. The different series of observations are combined 
with each other to form a correlation matrix, which is then ana- 
lyzed; the whole procedure is conducted according to certain 
mathematical rules. The factor loadings obtained are deduced 
from the observed values. This procedure, in itself, only gives 
the factors descriptive properties. If, besides this, they are to 
have an explanatory meaning, then the analytical method or the 
further steps which follow must be so constituted, that the factors 
obtain also some new characteristics typical of explanatory con- 
cepts. Such characteristics are, for example, simplicity, invari- 
ance, and conformity to results obtained by other methods of 
observation and inference, 
| The meaning of explanatory factor analysis is unclear. This 
is due in part to the fact that much in the terminology, e.g. the 
factor concept itself, is ambiguous and used in varying contexts. 
Furthermore, statements made by different scientists on scientific 
theory very often imply differences of opinion. Both in psycho- 
logy and in the general theory of science 
still being debated. In the next chapters in 
to determine, on the basis of this debate, w 


given to explanatory factor analysis, and wh 
conduct such analyses, and w 


these questions аге 
ttempt will be made 
hat meaning can be 
ether it is possible to 


hat methods should then be applied. | 


CHAPTER IV 


FACTOR ANALYSIS AND THEORY OF SCIENCE 


In the behavioral sciences there is a need for a firmer founda- 
tion for research methods and for development of scientific 
theories and laws. Some of the attempts made to build a founda- 
tion are known as the theory of science or the philosophy of 
science or the science of science, and are greatly influenced by 
modern philosophy, for example by the school of logical empiri- 
cism. In the following pages, an attempt will be made to utilize 


some of the results of the theory of science, which are relevant 
to a discussion on the meaning of, and the distinction between, 


descriptive and explanatory factors. Even if much of the following 
discussion may prove to be one-sided and oversimplified, we 
still hope that it may nevertheless have a constructive contribu- 
tion to make, by arranging the problems, and suggest solutions, 
in a way which will enable subsequent investigators to solve 
them. 
One of the typical features of the theory of science is, that it 
takes the methods and results of empirical investigation as its 
Starting point, and then endeavours to clarify their meaning and 
implications. Hence, it is not mainly an isolated attempt to ad- 
vance cosmological theories, in the form of general theories of 
nature, but a concentrated endeavour to analyze the methods and 
theories which are developing in the different sciences. Of 
particular interest in this connection are the logico—semantic 
analyses of scientific theories and their elements. 4 
Two closely related fields of investigation will now be consid- 
ered. They both have an intimate bearing on the meaning of, 
and the distinction between, descriptive and explanatory factors. 
The first, is the general debate, within the theory of science, on 
the distinction between descriptive and explanatory models, and 
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the second is the discussion carried out mainly in the Psycholog- 
ical Review, concerning intervening variables and hypothetical 
constructs. 


Description and Explanation in the Theory of Science 


One of the tasks of science is to establish laws governing empir- 
ical facts. These laws should connect observations of separate 
facts into a logical system, making possible the communication 
of knowledge and prediction of new facts and events (Braith- 
waite, 1953, p. 1). The ideal would be if all established regulari- 
ties and other facts could be deduced, as logical consequences, 
from a few universal laws. A system of such laws constitute a 
scientific theory that is said to be an explanation of the empirical 
facts. A scientific theory thus consists of a number of terms linked 
together to propositions or formulae through a language of logical 
or mathematical nature, in such a way that the empirical facts 
can be deduced from these terms. These formulae and proposi- 
tions should be of such a nature that they can form a basis for 
hypotheses which can be empirically confirmed. 

To avoid “metaphysical” infusion into the theories, some 
Scientists, e.g. among the operationists, maintain that scientific 
theories ought to be based on concepts and laws which are de- 
duced entirely from facts empirically established (Marx, 1951). 
This opinion has been criticized by e.g. Carnap (1956) and Hempel 


(1952). The latter says about theories deduced from empirical 
data (p. 23-24): 


'The conjecture mentioned in the preceding paragraph may now be 
restated thus: Any term in the vocabulary of empirical science is de- 
finable by means of observation terms; i.e. it is possible to carry out à 
rational reconstruction of the language of science in such a way that all 
primitive terms are observation terms and all other terms are defined by 
meang of them. This view is characteristic of the earlier forms of posi- 
tivism and empiricism, and we shall call it the narrower thesis of empiri- 
cism. According to it, any scientific statement, however abstract, coul 
be transformed, by virtue of the definitions бё its constituent technical 
terms, into an equivalent statement couched exclusively in observation? 
terms: Science would really deal solely with observables. ... Despite 


its apparent plausibility, the narrower empirist thesis does not st? 
up under closer scrutiny. 


B 
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The main question is whether there is any difference in prin- 
ciple between explanation and description. Here we have limited 
"explanation" to mean that the special facts can be deduced from 
general concepts. This general qualification is found, for instance, 
in a paper on the meaning of scientific explanation written by 
Feigl (1953). In this paper the meaning of description is also 
discussed. This is illustrated by the following quotation (p. 511): 


It is very helpful to restrict the meaning of “description” to singular 
Statements representing fully specific facts, events or situations. Such 
descriptions may appear as the conclusion of explanatory, i.e. deductive 
inferences. Some of the premises of these inferences must then be scien- 
tific laws or theoretical assumptions. Since laws and theoretical assump- 
tions are (or at least contain) generalized statements (i.e. unlimited 
universal Propositions) they are not classified as descriptions. 


These distinctions between explanation and description are 
probably generally acceptable but there are different opinions 
as to what should be included in the concept of “generalized 
statement” in the definition of explanation. If the generalized 


where explanation on the whole referred to more “singular 
statements, representing fully Specific facts”, than description- 
ntioned, some scientists 


ness of meaning, and make Possible the formulation of other laws 
and theories (Carnap, 1956, РР. 66-69). This assigns, to these 
constructs a surplus meaning over and above that which can be 
deduced from the Specific facts. In Principle, this introduces 
something new at least into certain explanations, besides what 
is included in description. Í 

The next question which has partly been touched upon, is 
concerned with what characteristics an explanatory concept 
should have, in order to be acceptable and useful, The require- 
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ments which are mentioned in the following pages, and which 
are taken from the field of the theory of science partly overlap. 
We shall not try here to separate them into logically independent 
concepts, or to determine which are the most important. 

Р Explanatory concepts ought to have the following characteris- 
ics: 

(1) Comprehensiveness.—Which means that the concept can 
be used in the deduction of many empirically established regulari- 
Чез and relations. For example, the concept of wave in physics 
brings together a number of optical, acoustic, and electrical 
Phenomena under one common theory. 

(2) Openness of meaning.—This is related to the requirement 
for comprehensiveness, and emphasizes that the concepts ought 
to be capable of development, so that new testable hypotheses 
can be deduced from the concepts and new facts fitted into 


still more general theories. 
It is mentioned above, that, for example, Braithwaite and 


Hempel have tried to show that only theoretical constructs are 
capable of development because they have surplus meaning. A 
theoretical construct however, is not only a metaphysical concept, 
since it is connected to empirical data by epistemic correlations 
which make confirmation possible (Northrop, 1947, pp. 119-131). 

(3) Simplicity.—This requirement is often emphasized in 
Scientific theory construction. The notion of simplicity implies 
that a theory should be made up of as few logically independent 
Propositions and theorems as possible (Carlsson, 1949, pp. 132- 
143). Each term must therefore include a large number of 
Phenomena and form an integral part of many theories. 

(4) Completeness.—An explanation also ought to give a correct 
and rather full description of the facts that are to be explained. 
This may conflict with the requirement for simplicity and may 
make it necessary to revise explanatory concepts when new and 


More exact data become available. | | 
(5) High degree of confirmation.— his signifies that it should 
€ possible to deduce a great number of empirically confirmable 
Conditions from the explanatory concepts. These confirmable 
Conditions ought to be such that, in principle, the concepts can 


б — 576055 S. Henrysson 


82 


be verified or disproved and may also be compared with alter- 
native explanatory concepts. 

In many connections, it is required that explanatory concepts 
should have "reality" or "meaning". These terms readily ac- 
quire a metaphysical tinge, and in the theory of science there is a 
marked tendency to replace them by the requirement for con- 
firmation (Carnap, 1956). 

(6) Invariance.—A. useful concept should be derivable and 
confirmable in many independent ways. This leads to the re- 
quirement that the content of the concept must be the same in 
different connections and in various empirical systems. 


Intervening Variables and Hypothetical Constructs 


The current trend in psychology is characterized by an en- 
deavour to link up different, empirically established, facts into 
strict psychological theories. The works by Tolman (1932) and 
Hull (1943) contain examples of such constructions of theory. In 
these and similar publications there is a definite tendency notice- 
able to give the theories, as far as possible, an operationally 
definable content. At the same time, theories should be simple 
and clear, which means that the variables should be few and 
fundamental in character, each comprising a number of important 
conditions for the behaviour studied. Tolman termed such 
variables intervening variables and required that they should be 
deducable from operationally definable conditions (Tolman, 1936). 

Difficulties were experienced in connection with these at- 
tempts. Concepts not fully reducible in accordance with opera- 
tionally definable conditions were sometimes included in the 
theories. Nor was the meaning of the concept intervening vari- 
able made fully clear. Hull, for example, introduced intervening 
variables in his theory in such a way that they sometimes seeme 
to be synonyms of what he called "hypothetical entities" (Hull, 
1943, pp. 21-22). 

MacCorquodale and Meehl (1948) have attempted to clarify 
this question in a paper in the Psychological Review, “Оп а 
distinction between hypothetical constructs and intervening 
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variables". In this paper they tried to limit intervening variables 
to the operationally definable sphere and to give other properties 
to the concept hypothetical construct. Their aims are clearly 


stated in the summary of the paper, as follows: 


1. At present the phrases "intervening variable” and "hypothetical 
constructs" are often used interchangeably, and theoretical discourse 
often fails to distinguish what we believe are two rather different no- 
tions. We suggest that a failure to separate these leads to fundamental 
confusions. The distinction is between constructs which merely abstracts 
the empirical relationships (Tolman's original intervening variables) 
and those constructs which are hypothetical" (i.e., involve the supposi- 
tion of entities or processes not among the observed). 

2. Concepts of the first sort seem to be identifiable by three character- 
istics. First, the statement of such concepts does not contain any words 
Which are not reducible to be empirical laws. Second, the validity of the 
empirical laws is both necessary and sufficient for the “correctness” 
of the statement about the concepts. Third, the quantitative expression 
of the concept can be obtained without mediate inference by suitable 
groupings of terms in the quantitative empirical laws. i 

3. Concepts of the second sort involve words not wholly reducible 
to the words in the empirical laws; the validity of the empirical laws is 
not sufficient condition for the truth of the concept, inasmuch as it 
contains surplus meaning; and the quantitative form of the concept is 
not obtainable simply by grouping empirical terms and functions. е 

4. We propose a linguistic convention in the interest of clarity; t г} 
the phrase intervening variable be restricted to concepts of the first kind, 
in harmony with Tolman’s original definition; and that the phrase 
hypothetical construct be used for those of the second kind. Р 

5. It is suggested that the only rule for proper intervening mp 
is that of convenience, since they have no factual content surplus to 
the empirical functions they serve to summarize- RI 

6. In the case of een fni constructs, they have a com 
factual reference in addition to the empirical data which ато О 
Support. Hence, they ought to be held to a more stringent ae se 
in so far as our interests are theoretical. Their actual =з а 
Compatible with general knowledge and particularly m dress 
relevant knowledge exists at the next lower level on the exp 


hierarchy, 
i eral 
Discussions have been conducted along these lines. Sev 


Contributors, for example, Bergmann (1953) and Maze (1954) 
doubt if it is really possible to maintain the distinction between 
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intervening variables and hypothetical constructs and if it is a 
fruitful one in scientific theory construction. Bergmann (1953, 
pp. 445-446), for example, emphasizes that as soon as inter- 
vening variables are included in a theory, they automatically 
acquire both surplus meaning and the status of hypothetical 
constructs. This criticism has been dealt with for example, by 
Rozeboom (1956), who has tried to define more precisely the 
meaning of intervening variables and hypothetical constructs, 
and the distinction between them. He emphasizes that an inter- 
vening variable is derived from certain empirical data through 
an analytical relation (p. 253): 


Ап intervening variable (transformation variable) is a variable which 
has been systematically defined in terms of its antecedents and is hence 
dependent upon the latter for its meaning. 'То recapitulate the analysis 
by means of a formal example, let us suppose that 2 = (x + y?)/w is 
an empirical relation expressing the best estimate of a known variable 
z as a joint function of other known variables ш, x and у. We are now 
able to express z as a function of analytical mediation variables by de- 
fining one or more of these as mathematical functions of w, x and y. 


Such an intervening variable, can, if it is appropriately chosen, 
combine certain empirical data into useful and operationally 
definable concepts. These concepts do not have any surplus 
content besides that which is derived from the empirical relation 
they are based on. Hypothetical constructs on the other hand, 


have such a surplus content, and are delimited in the following 
way (op. cit., pp. 254-255): 


Hypothetical constructs—In the last section we considered those 
mediation variables which derive analytically from their antecedents- 
For the remainder, passage from antecedents to mediators must be made 
with less than logical certainty, and such variables we shall designate 
as "inferred variables". Precisely because inferred variables are not 
analytically contained in their antecedents, it follows immediately 
that (a) the meaning, or ontological content, of the inferred variables 
is not reducible to that of its antecedents, and (Б) a certain degree 0 


error must be expected in passage to the inferred variable from its 
antecedents. 


Roseboom thinks that the notation "hypothetical" for these 
constructs has overemphasized their unverified nature. Many 
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concepts, which are used in scientific theory and which are well 
confirmed, are, in his opinion, still to be considered as hypothet- 
ical constructs in principle (op. cit., p. 255): 

I suggest that by "hypothetical construct” we understand all cases 


of inferred mediation variables, even though in a few instances the 
generic existence of all the variables and its applicability to the objects 


concerned may admit of little practical doubt. 

A similar position is taken by Ginsberg (1954). He especially 
emphasizes that concepts relevant to areas other than those where 
they are operationally defined, aquire the status of hypothetical 
constructs. 

An operationally oriented scientist like Marx (1951) accepts 
hypothetical constructs as useful tools in the process of building 
a psychological theory, but he considers them most useful in the 
preliminary stages of scientific research in a special field. In this 
connection, they can give rise to fruitful hypotheses for empirical 
investigations, and thus be converted into operationally valid 
intervening variables. In his opinion, only operationally definable 
concepts are finally acceptable in a sound scientific theory. 

The main impression gained from reading the literature on the 
Subject is, that hardly any scientist denies that hypothetical 
Constructs have their part to play asa link in psychological theory 
Construction, and even Tolman (1951, p. 282) has accepted 
them. Here, we do not have to determine to what extent it is 
both possible and desirable to try to reduce them completely to 
Operationally definable concepts. However, even if one does not 
wish to distinguish between intervening variables and hypothet- 
ical constructs, one nevertheless has to accept the existence of 
both of some kind of descriptive concepts which are opera- 
tionally definable and theoretical concepts with surplus meaning. 

It is difficult to summarize the discussion, which apparently 
is far from finished. If intervening variables, or whatever we 
may choose to call them, are considered as variables which are 
fully reducible from an empirical relation, it would appear that 


these concepts are insufficient for the construction of scientific 
theory, Concepts which are not fully operationally defined are 
s satisfied by hypothetical 


also required. This requirement 1 
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constructs which have the comprehensiveness and openness of 
meaning, which make them useful in scientific theories that cover 
a number of regularities obtained empirically and that form a 
basis for hypotheses as to new theories. Scientific theories, which 
work with such hypothetical constructs, have, in most sciences, 
been very fruitful also for further empirical research. 


The Distinction between Descriptive and Explanatory 
Factor Analysis 


The review of the discussion on the theory of science indicates 
that it is fruitful to make a distinction between descriptive 
concepts deducible only from empirical data, and, theoretical 
constructs with surplus content and openness of meaning. The 
latter enable us to formulate a number of propositions and the- 
ories, which can be confirmed by many different methods on а 
wide range of data. 

Accordingly, such factors should be called descriptive, as are 
operationally defined by the use of a specific method employed in 
a specific test battery and administered to a specific population. 
Those factors may be called explanatory which have surplus 
content referring them to factors found in other batteries and 
populations. А number of other confirmable propositions and 
theories can be deduced from this type of factors. 

Hartley (1954) has tried to make a distinction between de- 
scriptive, and what he calls inferential factor analysis, and has 
discussed how the latter should be conducted. He delimits 


descriptive factor analysis in the following manner (op. сї. 
рр. 196-197): 


"Thus, by descriptive factor analysis is herein meant the kind of factor 
analysis that aims at converting the correlation matrix to a more usable 
form. The facts the descriptive factorist wishes to deal with are repte- 
sented solely by the correlation matrix. His factor analysis, like the 
draftsman's drawing, adds no factual material but contents itself with 
an exact restatement of the factual material expressed by the correlation 
matrix. Hence terms like “right”, "wrong", "error", “true”, **false" 
are inapplicable to the factors of descriptive factor analysis. 
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There are factor studies which, besides a purely descriptive 
presentation, attempt to give properties to the factors, referring 
to such concepts as “underlying unities". Hartley gives the fol- 
lowing definition of this type of factor analysis (op. cit., p. 197): 


It may be possible, then, to characterize adequately inferential factor 
analysis as that form of factor analysis in which the factor matrix has 
empirical meaning in addition to that expressed by the correlation 
matrix. Two different factorial solutions for the same correlation matrix 
could, then, express different and even contradictory meanings. The 
meaning of the factor matrix which is in addition to that expressed by 
the correlation matrix is the reference to the unobserved but hypothe- 
sized “factors”. Since these factors are not referred to by the correlation 
matrix, the correlation matrix would seem to be serving as the evidence 
from which the factors are inferred by some process or other. Hence 
the term “inferential” has been selected to characterize this form of 


factor analysis. 


When we wish to define our attitude towards the question 
whether inferential factor analysis is possible, it is, according to 
Hartley, first necessary to find out if such factor analytical terms 
as “underlying unities” have any definite meaning. Hartley deals 
with this problem in his unpublished doctoral dissertation (Hart- 
ley, 1952). After making a logical analysis, Hartley finds that it is 
Possible to formulate a language which gives such properties to 
the factors. This language covers also new properties of the fac- 
tors besides those deduced from the data in a specific investi- 
gation. For factors which possess inferred properties it is SIg- 
nificant to ask whether they are “right” or “wrong”. 

When Hartley discusses criteria for judging whether the factors 
are “right”, he refers to Carnap’s discussion of scientific theories. 
Carnap (1950) points out that an important criterion of з 
validity of scientific theories is their degree of confirmation. The 
degree of confirmation of a theory is higher, if it can be verified 
by different empiricial investigations and if it can form the b 
for new testable propositions. From this point of view, Hartley 
(1952, p. 110) gives the following criterion: 

eld i is as herein conceived, should accept as 
its a wea LM Een having the highest degree of confirma- 


tion on the evidence present. 
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One way of obtaining a higher degree of confirmation is to 
make several factor studies of samples from different populations 
and different tests. This is the same as making studies of invari- 
ance. 

Hartley believes that inferential factor analysis, based on the 
principle of simple structure, has a higher degree of confirmation 
than others types of factor analysis. This is, because it is based 
on the principle of parsimony. He finds it difficult to show 
this, however, because the principle of simple structure is so 
vaguely formulated. 

Hartley's distinction between descriptive and inferential factor 
analysis seems fruitful in the light of the discussions on the 
theory of science which have been reviewed above. 

When we try to interpret the factors found through factor 
analysis and to give them an empirical basis, which makes the 
range of their validity wider than that of a certain constellation 
of tests and a certain population, and which includes them in а 
more general psychologica] theory, then we provide them with 
surplus meaning and thus convert them into hypothetical con- 
structs. Factors with such Properties are useful as explanatory 
Concepts in psychological theory, since they are obtained by 
different factor analyses and various kinds of investigations of 
different types of tests and populations. A consequence of this 


is, that explanatory factors should have the property of being 


that explain as many regularities and relations as possible, even 
outside of an actual factorial study, 


CHAPTER V 


EXPLANATORY FACTOR ANALYSIS 


The Testing and the Generating of Hypotheses 


. We may distinguish between two ways of using factor analysis 
in an inferential procedure for finding explanatory factors, 
namely, for testing hypotheses and for generating hypotheses 
(Eysenck, 1953). 

Аз has already been shown in the historical s 
It, psychologists, such as Spearman, Holzinger, Burt, and Thom- 
son, mainly limit their use of factor analysis to the testing of 
hypotheses already formulated. They employ a procedure where 
one or more factor analyses, together with other information, 
will show whether hypotheses, previously advanced, as to the 
Nature of the underlying processes can be confirmed or not. 

In such a procedure, hypotheses as to structure, are tested on 
Various sets of data. If the factor structure cannot be applied to 
the data, the hypothesis can be rejected. If, on the other hand, 
the analysis supports the hypothesis, this is an indication of 
the correctness of the hypothesis. Conformity, however, is only 
to be considered as a necessary, and not as a sufficient, basis for 


the correctness of the hypotheses, since many other hypotheses as 
be concordant with the 


to factorial structure may also prove to ; 
alysis, like the testing of 


nature of the data. This type of factor an i 
back that a hypothesis can 


hypotheses in general, has the draw 

be rejected but it cannot be proved true and thus enable us to 
exclude other hypotheses. The only thing which can be safely 
Said, when empirical data are found to support a hypothesis, is, 
that the hypothesis gives one of many possible descriptions of 
data. If the present hypothesis as to the factor structure, should 
also be advanced to explain why the relations between the obser- 
Vation series have certain properties, then also other requirements 


urvey in Chapter 
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for the validity of hypotheses must be induced. These require- 
ments ought to be such, that as many alternative hypotheses as 
possible are rejected. This necessitates factor analyses of various 
populations and with differently composed test batteries and even 
entails investigations or considerations other than those of a 
purely factor analytical nature. 

The testing of hypotheses by means of factor analysis can also 
be combined with procedures that differ from those which vary 
population or test battery. One can try to study, for example, 
changes in the supposed factor by experimental variation, growth 
and training. The influence the factor is exposed to, should then 
change its loadings in the same proportion in all the variables 
where the factor is included, while the loadings of the other 
factors remain unchanged. For example, in a brain-surgery 
operation that reduced the variance of a memory factor by half in 
one test, the loadings of this factor should also be reduced pro- 
portionally in other tests. 

The fewer, the hypotheses which are tested together at the 
same time, the safer their confirmation, as a rule. It is often 
possible to get the results of a factor analysis to correspond to 
those required by certain of the hypotheses, if we capitalize ОП 
errors of sampling and measurement. The observed variables 
should consequently be selected in such a way, that the principal 
features in the common variance of the battery, which are not 
being studied, are known through earlier results. If part of the 
structure is already well established in this way, one can study 
with more confidence whether the hypothesized factors are really 
found in the tests, where they should be loaded according to the 
hypotheses. 

When operational hypotheses are tested by factor analysis; 
this forms only a link in investigations, which should include 
other studies, for example, of a Clinical, genetical, or experimental 
nature. 'T'he greater the number of operational hypotheses which 
can be advanced and verified within the limits of a main hypothe- 
sis as to an underlying factor, the more reason there will be for 
maintaining that the factor is “fundamental” and indicates 5000 
basic and universal characteristic of the underlying structure. 


> 
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— dd сен. it may be said that hypotheses can be tested 
y of factor analysis. Whether the factors that pass such 
ypothesis-testing are to be given an explanatory meaning or 
ж will then also depend even on the extent to which Kod 
ГЕ" exists that the factors possess explanatory character- 
м € possible ю maintain any clear distinction between 
"s g and generating of hypotheses. By hypothesis-testing 

me of the hypotheses can be disproved, and at the same time 
a basis can be found for establishing of new ones. These, in their 
turn, require new tests, and so on. Nor is the generating of 
hypotheses a completely isolated procedure. It is usually based 
оп some form of more or less implicit hypotheses, which become 
evident in the selection of variables and in the choice of the type 
of solution. In any case, the factor analyst must work with the 
hypothesis that some form of structure exists, and that this has 
certain characteristics, in regard to its general type, which make a 


unique solution possible. 


Hypothesis-Generating Factor Analysis and the Principle 
of Parsimony 


On the whole, it is only Thurstone and his followers who 
make a serious attempt to assign to factor analysis а hypothesis- 
generating role. Consequently, the discussion on the possibilities 
of generating hypotheses by means of factor analysis may be 
considered in connection with Thurstone’s approach. His ambi- 
tions are evident from the following statement (Thurstone, 
1947, p. 55): 
by means of individual 
ys. One can invent a 
hat underlie the individual differ- 


1 experiment, or à more direct 
s. If no promising hypo- 


ain is to be investigated 


When a particular dom: 
e of two wa 


аа one can proceed in on 
ypothesis regarding the processes t 
ence > 

nces, and one can then set up 2 factorial 


laboratory experiment, to test the hypothesi 
thesis is available, one can represent the domain as adequately as 


Possible in terms of a set of measurements of numerical indices and 
Proceed with a factorial experiment. The analysis might reveal an under- 
lying order which would be of great assistance in formulating the scien- 
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tific concepts covering the particular domain. In the first case we 
start with a hypothesis that determines the nature of the measurements 
that enter into the factorial analysis. In the second case we start with no 
hypothesis, but we proceed, instead, with a set of measurements or 
indices that cover the domain, hoping to discover in the factorial analysis 
the nature of the underlying order. It is this latter application of the 
factorial methods that is sometimes referred to as an attempt to lift our- 
selves by our own boot straps, because the underlying order in a domain 
cannot be discovered without first postulating it in the form of a hy- 
pothesis. This is probably the characteristic of factor analysis that gives 
it some interest as general scientific method. 


This latter type of factor analysis, also called explorative factor 
analysis, is to be used primarily in the mapping of a field about 
which we have little knowledge or developed theories. The 
results of such analysis can then be used for formation of more 
rigorous hypotheses and in planning experiments. 

As is pointed out in Chapter II, Thurstone’s intentions should 
be seen against the background of his pragmatical attitude to 
science. He starts out from the fact that alternative descriptive 
systems are possible and considers that in the choice between 
these, the principle of parcimony should be decisive. This prin- 
ciple should then be applied within the whole field that the scien- 
tific theory is to cover and to all empirical facts that can be deduced 
from the theory. This broad application of the principle of parci- 
mony has important consequences. It means, for example, that 
the number of concepts should be few and invariant in regard 
to their content in all their connections. Furthermore, simplicity 
requires that these concepts shall be applicable in as many 
connections as possible. The mathematical methods used by 
Thurstone to find factors satisfying these requirements are 
also based on the principle of parsimony (Thurstone, 1947, 
p. 333). This means that the number of common factors used for 
representing the correlation matrix should be as few as possible. 
Moreover, each factor should be loaded only in a few tests accord- 
ing to certain rules. 

Thurstone wants to find, by means of the principle of parsi- 
mony, a few invariant factors which are comprehensive in the 
development of scientific theory. By applying the principle d 
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parsimony, Thurstone considers that he has avoided metaphys- 
sally oriented discussions whether the factors are "real", 
existing", etc. 

Even in explanatory factor analysis it is necessary to be restric- 
tive in the selection of the problem. This can be done by selecting 
the tests so that large parts of the factor structure of the battery 
are already known and by making the number of tests large in 
comparison with the number of factors in the battery. The aim 
is to make the factor pattern as overdetermined as possible. 
This can be achieved if each factor gets loadings in three or 
possibly more tests, at the same time as the requirement for 
Simple structure is fulfilled. Consequently the selection of the 
tests should be based on a rough idea of how many factors have 
to be dealt with, and in which directions they can probably be 
localized. Thurstone therefore emphasizes, that it is not possible 
to find a meaningful structure in any arbitrary collection of tests 


(op. cit., p. 313): 


In planning a factorial analysis it is desirable to cover the domain 


to be investigated as completely as possible with a large number of 
related variables, which should be experimentally independent but 
Correlated and representative of different aspects of the domain. When 
а study has been planned this way, it will nearly always happen that 
every test vector lies near some other vector in the system. This arrange- 
Ment is desirable for several reasons, principally in order to make it 
Possible to discover a simple structure if it exists in the domain. 


is any simple structure 


Whether, and to what extent, there j 
the analysis will have to 


in the field is an open question, which х 
answer, In an explorative study of a completely new field it 
often happens that only a part of the structure is overdetermined 
and that in a more or less obscure way- But this, however, can 
form the basis for the design of new factor analytical studies or 
for investigations of a different nature, that lead to a safer deter- 
mination of the factors and their characteristics. In this way 


explorative investigations lead, step by step, over to the vei 
a hypotheses. Factor analysis, according to the principle о! 
Simple structure, is intended to afford material for the formula- 
Чоп of a number of more strictly defined hypotheses as to par- 
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simonious, unique, and invariant structures which should then 
be further tested and modified. Thurstone thinks that such a 
gradual procedure is the proper way of obtaining factors repre- 
senting essential aspects of the underlying structure, since the 
procedure is based on the principle of parsimony, which has 
been successful in other sciences. On this point, views differ, and 
both the theoretical and mathematical bases of the simple struc- 
ture solution have been called in question. Thomson (1948) and 
Reyburn & Raath (1949) have, for example, discussed these 
problems critically. 


а = y L 


CHAPTER VI 


FACTOR ANALYSIS ACCORDING TO THE 
PRINCIPLES OF SIMPLE STRUCTURE 


The Meaning of Simple Structure 


The principles of simple structure are fundamental for Thur- 
Stone's methods and make it possible to find factors with a 
range beyond pure description. The principles of simple struc- 
ture have been advanced to meet the requirements for parsimony, 
Invariance, uniqueness, and functional unity, which have been 
Previously mentioned. 

Restrictive assumptions must be made in order to get a unique 
Solution, The series of restrictions included in the principles of 
Simple structure are based mainly on the principle of parsimony. 
In this connection, the latter principle means that every test 
should have as simple a factor pattern as possible, and conversely, 
that the estimation of each factor should need a combination 
ОЁ only a few of the tests included in the battery. These require- 
Tents must be stated more precisely to make a unique and invari- 
ant solution possible. Thurstone tries to do this in the follo 
Way (op. cit., p. 320): 


wing 


If an analysis is to be made by the principle of simple structure, then 
the investigator gambles that the complexity of each test or measure- 
Ment is less than the complexity 7 of the battery as а whole. He hopes 
that the test complexities are considerably less than that of the battery. 
If the test complexities are less than 7, then there will be one ot di" 
lank cells in each row of the factor pattern. Such a pattern He à 
considered as evidence of an underlying simple order in which eac! 
Parameter or process in the domain is not involved in every test measure- 
ment, 
The above requirement of a complexity in each variable, which 
es a more strict formu- 


15 less than the rank r in the matrix R, mak 
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lation of the principle of simple structure possible. Its meaning 
can be illustrated by the vector theory, mentioned in Chapter I. 
The different tests included in a test-battery have their counter- 
part in test vectors. Since the factor analysis only applies to the 
common part of each of the т number of tests, each test vector 
will have a length equivalent to h, i.e. the square root of the com- 
munality 4. The correlation between the tests can be expressed 
as functions of the angles between the test vectors and of their 
length, according to the formula 


Tj — hh, cos by, t= ppa jt). 


$z is the angle between the vectors h; and h. This theory is 
given in more detail by Holzinger & Harman (1941). 

The test vectors in a test battery form a cluster of vectors, with 
certain angle relations to one another in a space with as many 
dimensions as the rank of the correlation matrix. The aim of 
the analysis is to refer this cluster of test vectors back to a system 
of reference vectors that fulfils the requirement for simple struc- 
ture. According to the requirements for such a structure none 
of the tests should be loaded with all the primary factors. In 
terms of vectors this means that the reference vectors should 
be chosen so that each test vectors has zero-projection on at least 
some of these reference vectors. 


Fig. 4. 
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ed en uis aer illustrated with three factors. The 
E € m ee in Fig. 4 by a cluster of arrows starting 
po ng three fan-shaped planes АОВ, BOC, and 
P ee p planes are defined by three reference vectors A,, 
"y e = which are normals, i.e., form right angles to each 
ann an 'The plane АОВ, for example, includes the test 
B ; 2, 3, and 4, which are at right angles to the reference 
ES or / z and thus have zero-projection on it, but show certain 
en pm on А, and A, The test vectors in the plane 
TA ave zero-projections on the reference vector A, and the 
ў ectors in the plane AOC on the reference vector A,. The 
cone of the test vectors to the different reference vectors 
pus to the required factor loadings. In this example, 
ors will have loadings according to the following table. 


TABLE 4. 
CENE cit el 
Factor 

Test & y z 
_————— 
1 0 x x 

2 0 x x 

3 0 x x 

4 0 x x 

5 2 0 x 

6 x 0 x 

y x 0 x 

8 x 0 x 

9 x x 0 
10 x x 0 
11 х x 0 
12 x x 0 
13 & x 0 
A 0 Е 0 
B 0 0 Ez 
E x 0 0 
Yee 


wo planes will have zero- 
forming normals to this 
ly on the third 


A test vector in the intersection of t 
rect on the two reference vectors 
Plane, 'This test vector will get projections on 


7 — 576055 S. Henrysson 
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reference vector and will consequently measure only one factor. 
The test vectors АО, BO, and CO are characterized by the fact 
that each of them is a measure of only one factor. When standard- 
ized in such a way that the communality is equal to one, they are 
called primary vectors, which, under certain circumstances, can 
refer to primary factors with explanatory properties. 

As a rule, the test vectors incorporate r dimensions; and thus 
r reference vectors are needed. The two-dimensional planes, in 
the three-dimensional case, correspond to what are known as 
hyper-planes of r-1 dimensions in the r-dimensional case. 

Ifr is greater than three, the vector system cannot be illustrated 
in the same way as in Figure 4. Multi-dimensional geometry 
and matrix algebra can be used, however, in cases where the 
rank is higher than three. 

In carrying out a factor-analytical investigation, we have to 
start with a cluster of test vectors defined by the correlation 
matrix. 'The primary vectors are the unknowns that are to be 
determined. The purpose of the analysis, according to the 
principle of simple structure, is to find a set of hyperplanes, 
each of the dimensionality 7-1, which at the same time include 


many test vectors. Thurstone formulates this as follows (op. cit., 
p. 328): 


Ther linearly independent reference vectors define the same number 
of co-ordinate hyperplanes. Each hyperplane is a subspace of dimen- 
sionality (7—1), and it is defined by a reference vector. The subspace of 
(r-1) dimensions which is orthogonal to the reference vector Ap will be 
called the co-ordinate hyperplane Г... If a reference frame can be found 
such that each test vector is contained on one or more of the r co- 


ordinate hyperplanes, then the combined frame and configuration is 
called a simple structure. 


Whether a factor model, that meets the requirements for 
simple structure, is applicable in a certain domain must be de- 
cided in each individual case, but Thurstone considers the 
principles to be so general, that they ought to have wide appli- 
cability (op. cit., p. 58): 


Our work in the factorial study of the human mind rests on the 
assumption that mind represents a dynamic system which can eventually 
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be understood in terms of a finite number of parameters. We have 
assumed, further, that all these parameters, or groups of parameters 
are not involved in the individual differences of every kind of mental 
task. Just as we take it for granted that the individual differences in 
visual acuity are not involved in pitch discrimination, so we assume 
that in intellectual tasks some mental or cortical functions are not in- 
volved in every task. This is the principle of "simple structure" or 

simple configuration" in the underlying order for any given set of 
attributes. 


‚Мо one has been able to show how the requirements for 
simple structure should be formulated in order to give a unique 
Solution. In the absence of a strict mathematical solution of 
this problem, Thurstone has, besides the minimizing of the total 
number of factors, proposed the following five criteria, which 
he has found by experience to give a good basis for the establish- 
ment of unique simple structure (op. cit., p. 335): 


We shall describe five useful criteria by which the r reference vectors 


can be determined. These are as follows: 
(1) Each row of the oblique factor matrix V should have at least one 


Zero. 

‚ ©) For each column p of the facto 
distinct set of r linearly independent tests w! 
Zero, 

(3) For every pair of columns of V there should be several tests 
Whose entries vj vanish in one column but not in the other. 

(4) For every pair of columns of V, a large proportion of the tests 
should have zero entries in both columns. This applies to factor problems 
With four or five or more common factors. 

(5) For every pair of columns there should preferably be only a 
Small number of tests with non-vanishing entries in both columns. 
That these five criteria are sufficient to give a unique solution 
has not been proved mathematically. On the contrary Reiersól 
(1950) has shown that they are not always sufficient, and has tried 
to formulate the requirements for a unique solution. Probably 
the five requirements are very often not sufficient for excluding 
Alternative solutions in favour of a specific solution. Thus, 
Bargmann (1951) has found that in many cases a number of 
alternative hyperplanes may exist. What can probably be done 
at present, is to determine the structure which, according to the 


г matrix V there should be a 
hose factor loadings vj, are 
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requirements for simple structure as now defined, affords to 
the best solution. In other words, we endeavour to determine 
the system of hyperplanes which is most overdetermined. 


Oblique Structure 


Spearman's original factor model included a general factor and 
one factor specific for each variable. The specific factors were, 
by definition, mutually uncorrelated and uncorrelated also with 
the general factor. When investigators became principally 
interested in finding group factors, the question arose whether 
they could be assumed to be correlated with one another or not. 

If factor analysis is employed only for finding purely descrip- 
tive factors suitable for representing a certain correlation matrix, 
then a model with uncorrelated factors is often easier both, to 
comprehend and to use. Moreover, uncorrelated factors are, from 
the viewpoint of mathematical statistics, considerably simpler to 
handle. To use uncorrelated factors in all cases would however, 
according to Thurstone (1947, p. 139), be a serious limitation: 


Among statisticians and psychologists there is a rather general belief 
that if human traits are to be accounted for by any kind of factors, then 
these factors must be uncorrelated, This belief has its origin in the 
statistical and mathematical convenience of uncorrelated factors and 
also in our ignorance of the nature of the underlying structure 0 
mental traits. The reason for using uncorrelated reference traits сап 
be understood, but it cannot be justified. Height and weight are two 
useful measures of body Size, even though they are correlated. If we 
Should use, instead, two linear combinations of height and weight which 
are uncorrelated in the general population, we might find such measures 
more convenient in statistical theory, in that the cross products would 
vanish and we could find a certain satisfaction in that the two measures 
are independent; but they would be awkward to think about. 


If one wishes to find primary factors referring to “functional 
unities", one has to take into account that these can be correlated 
with one another. They can be connected with each othet 
through, for example, the influence of heterogeneity of the 
population, or influence of growth. In a structure of abilities 
consisting, for example, of a numerical, a verbal, an inductiv® 
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and a spatial factor, it is fully possible that they are mutually 
connected with one another. In any case, the method of analysis 
must be open to this possibility. 

It is entirely reasonable that factors which are uncorrelated or 
low-correlated in one population will be more highly correlated in 
another population and vice-versa. If, for example, the structure 
of abilities in one population is relatively homogeneous with regard 
to the level of general intelligence, then the group factors of 
ability in this study could have a considerably lower correlation 
with one another than in a sample from a population with greater 
variations in the general level of intelligence. Examples of this 
are factor analyses of college students compared with those of 
representative samples from the entire population. If the factor 
pattern is to be invariant, it is necessary to allow for changes in 
the correlations between the factors. This has recently been 
pointed out also by the statisticians Anderson and Rubin (1956, 
p. 134). 

A drawback that has been emphasized, for example, by 
Reyburn-Raath (1949), is that oblique structure opens up pos- 
sibilities for still more alternative solutions than only orthogonal 


structure does. 
Whether correlated factors are to be accepted in a certain 


field is a question that will finally be settled by studies of the 


factors also in non 


The Selection of Test Battery 

When the structure in a certain field is to be investigated, all 
the available knowledge should of course be used. On this basis 
We may perhaps get an approximate idea of the number. of 
dimensions within the field and of the hypotheses regarding 


Some of the factors. Such hypotheses, on which the selection 
cit. A basic prin- 


Of tests is based, are always more or less impli 
i ү that the different 


Ciple in the selection of tests for the battery 15, 

Parts of the field should be covered by tests that are correlated, 

but at the same time, experimentally independent. A rule which 
sts loaded with a certain 


is often applied, is that more than two te 
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factor, are needed to locate the hyperplane related to this factor. 
This is done in order to make the factor overdetermined and 
easier to interpret. 

A major principle in the selection of tests is that none of them 
shall require all the dimensions that are included in the field. 
This is how Thurstone (1947, p. 179) expresses it: 

If the test battery is a hodgepodge of complex tests, there may not 
exist in the configuration any clues as to what the underlying functions 
may be; but this possibility is dependent on the intuitions of the experi- 


menter in assembling a significant set of measurements rather than on 
factorial theory and method as such. 


The results of the analyses will show, if the test has been 
selected in such a way that it meets the requirements for simple 
structure, and so that the solution becomes unique and invariant. 
If only certain factors become well defined, new hypotheses must 
be formulated as a basis for selection of new tests which will 
more clearly define the structure. It is necessary to have a series 
of factor analyses, of which the first are more explorative, while 
subsequent analyses deal increasingly with tests built on hy- 
potheses which are based on the results of the earlier analyses. 

Another reason why simple structure cannot be obtained may 
be that the test battery covers too small a field. If, within the 
field, there is a dominant functional unity that is present in al 
tests and that cannot be divided into further factors, then the 
field must be expanded. The test battery must have the property 
that none of the tests includes all dimensions. Otherwise the 
principle of simple structure is not applicable to this set of data, 
and another factor model must be selected. 

An often disregarded requirement is that the tests should have 
high or, in any case, known reliability. Since the factor loadings 
in a variable decrease in proportion to the square root of the 
reliability of a variable (Guilford & Michael, 1950), low relia- 
bility will result in the factor loadings in this variable becoming 
so low that they might be overlooked in the rotational procedure 
or in the interpretation of the factor structure. ; 

There is a mathematical relation between the factor табх 
A,, obtained in the analysis of the correlation matrix correcte 
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for attenuation R,, and the factor matrix A,, obtained in analysis 
of the correlation matrix without correction R. This relation 
follows the matrix algebraic formula 


А,= А.У, (68) 


where V is the diagonal matrix which contains reliabilities. This 
formula, as well as the following ones, are special cases of the 
general formulae given by Lawley (1940, pp. 69-70). 


(69) 


The relation between R, and R follows the formula 


R,- УВУ (70) 


Which is the general matrix formula for the well-known attenua- 
tion formula 
om. (71) 


Тао = 7 
Vra Too 


The above equations imply that one can, theoretically speaking, 
make a factor analysis of unreliable variables and, after correc- 
tion for attenuation of the matrix of loadings, obtain the same 
loadings as an analysis of fully reliable tests should give. This 
correction for attenuation should be made by dividing the load- 
ings of each unreliable variable by the square root of the relia- 
bility of each variable. In practice, the conditions can be more 
complicated. If, for example, two factors have the loadings of 


0.40 and 0.35 in a fully reliable test, those loadings сап, because 
drop to values that would be 


of the lack of reliability of the test : sld b 
disregarded in the rotational procedure. This can result in | : 
Totated axes being chosen in a quite different manner uut 
employed when factors based on fully reliable tests are TO A 
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Communality Estimates 


Communalities, instead of unities, are included in the diagonal 
of the correlation matrix, if the analysis is to be limited to the 
common part of the variances. At present, there is no fully 
satisfactory method for determining communalities. A mathe- 
matically acceptable solution seems to require that the rank of 
the matrix R is determined in advance (Rosner, 1948). A number 
of approximative methods have been suggested for estimating the 
communalities, before the analysis of the correlation matrix starts. 
"These more or less imperfect methods will not be discussed here. 
"They are discussed, for example, by Thurstone (1947, pp. 282- 
318). 

It should also be pointed out that the wider application of 
electronic computers will make it possible to find suitable com- 
munality estimates, through iteration, without entailing too much 
work. This iteration requires, however, a hypothesis as to the 
rank of the correlation matrix. . 

Lately, other methods have been added that do not require 
communality estimates made in advance. An example of this 15 
Lawley's method No. I. If a certain rank is assumed, Som 
munalities are obtained as bi-products of the analysis itself. 
Lawley's method, however, requires much work, unless elec- 
tronic computers are available. 

Whittle's (1952) methods, which will be discussed in pu 
detail in Chapter VIII, require only approximate knowledge 2 
the proportional size of the communalities in relation to еде 
other. How approximate this must be, is not known yet, since 
experience with the methods is rather limited. 

Thurstone, in cooperation with Bargmann and Henrysson, m 
tried to use the principal component method for correlation m 
trices with zeroes in the diagonal instead of communalitie 
(Thurstone, 1954 b, and Bargmann, 1955). © 

As an introduction to this approach, the following notes ad 
the principal component method for a correlation matrix W't 
communalities in the diagonal may be useful. 


———— LL 
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In a princi 
pal component soluti i 
EEDA р olution, the loadings must follow 


_ t=l H 
=. (р hhn d=1,2, ... m). (72) 


In thi DNE i 
s formula 7,, is a residual correlation after d-1 factors 


ha 
- ir taken out from the matrix R. 
e same time, the principal component loadings follow the 


relation 
(d,e=1,2, „m, d*e). (73) 


Ms 


4,40, —0 


t=1 


0 


Is А 
a аа instead of 7 is inserted in the formula 72, the fol- 


lowing formula will apply for loadings of the first factor 
n m m n 
È ац У ja ia У аа Уап Фа 
га о. a йл E y (74) 
> а > ай 
tel 


This formula can be divided in the following manner 


n m n 
аһ 2, а У аа 2, an Фа 
tel d=2 tel 
an= —5 + a . (75) 
X ah у аа 
tel tel 
п 
Sin x ай ï 
ES tl =1 and X an 470 
2 tel 
È а 
tel 
th 
€ final result is that ау ^ 4r i 
ethod of factor analysis 


ee main idea in Thurstone’s т 
Я ithout communalities is, to use 2 principa 
9 modified that the requirement j* t 1S пй 


1 component solution 
posed on the relation 
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G,—-—— (jt=1,2, set): 


This means that the communalities are zero in the matrix R. 
Through the addition of this requirement, one of the products 
л 


will disappear іп the product sum > a,, а, of formula 75, 
t=1 


namely, the опе for which t =]. This reduced product sum will 
no longer be equal to zero for the ordinary principal component 
loadings calculated from the matrix R with communalities. Prin- 
cipal components analysis, with and without communalities, will 
therefore give the same results only when the rank of the matrix 
R is equal to one. In this case, the term after the plus sign in 
formula 75 will disappear and there will only remain 


я А 
È anfa > ah 
an = =a san (jt-12,5j«0). (76) 
> а > а 
tel tel 


When the rank is higher than one, the differences between the 
loadings according to the two methods will often be large. 

In connection with a study of Lawley’s maximum likelihood 
method I, Henrysson (1950) constructed 12 artificial covariance 
matrices of nine variables, each containing 200 observations. 
The observations were based on numbers taken from Wold’s 
Random Normal Deviates (Wold, 1948). The covariance matrices 
have been converted into correlation matrices in connection 
with the following study. The expected structure of each vari 
able is very simple, containing one general and one unique factor 
according to the equation 


2 2 
Вац, (7=1,2...9; i=1, 2... 200) (77) 


E 


The correlation matrices are given in the appendix. po 
each of the 12 correlation matrices, one factor has been taken 23 
according to each of the principal component methods, with ап 
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with iti 
ie ы coran tux кы clau. 
а fa th —— ities, have been used as com- 
Ls > e met od w ith communalities. The results of 
Fig É ods, which are given in the appendix, agree closely. 
am e 12 x 9— 108 pairs of loadings, no loadings in a pair 
iffer by more than 0.001. 
bier E good agreement cannot be expected when the rank is 
pi an one. This is illustrated in a correlation matrix known . 
ср E two. Lawley and Swanson (1954, p. 76) have pub- 
s suc a matrix. If the theoretical values of the communali- 
es are used in the diagonal, the correlation matrix will be practi- 
cally empty after two factors are taken out by the principal com- 
er method with communalities. The loadings are to be 
E table 5. The residual correlations lie within the limits of 
ing errors, since none of them has a value over 0.001. 


T З 2 mE 
ABLE 5. Matrices of factor loadings after principal component 
analysis with and without communalities 


Loadings after 


Loadings after 
Vari analysis with " analysis without " 
ariable communalities h? communalities h? 
I II I II 
1 741 .560 .863 .693 .565  .799 
2 .869 351 .878 861 376 .883 
А 664 1670 1890 1600 1660.796 
4 .894 .003 .799 .933 017 871 
5 818 — 405 `833 `806 -391 -803 
6 813 -.483 .894 .785 —459 .827 
.700 -.550 __ .793 


——7 78 -593 91 
x 6.068 5.772 


munalities of the same matrix, the 


In the analysis without com 
s shown in table 5. The residual 


loadings are quite different, asi 


Correlations, after two factors are taken out, are larger in this 
se results illustrate what has 


жы as can be seen in table б. The h 
ееп shown algebraically, namely, that the two methods give 
the same result only when the rank is one 1n the correlation 


Matrix R. 
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TABLE 6. Residual matrix after two factors by principal component 
analysis without communalities 


.032 .078 .007 .041 .046 .048 


.032 . .047  —.030 .022 .034 045 
.078 047 024 .046 .047 .041 
.007  —.030 .024 —.015 .002 .023 
041 .022 -046 —.015 .049 .073 
.046 .034 .047 .002 .049 .093 


.048 .045 .041 .023 .073 .093 


In conclusion, the impression gained is that the principal 
component method without communalities is not very useful 
except for rank one. In this case, other simple methods are 
available, for example, Spearman's, which does not require 
knowledge of the communalities. 

The only method, which is useful when the rank is over опе, 
and which does not require communality estimates made in 
advance or any assumptions about their size, is, at present, 
Lawley's method I, which will be discussed in Chapter VIII. 


Analytical. Methods for Rotation to Simple Structure 


The analysis of a correlation matrix into a simple structure 
factor matrix must, at present, be made in two steps. First, 
preliminary factor matrix is determined, if possible by means ofan 
efficient method of analysis. This method should include a test 
of significance so that the order of the factor matrix can be kept 
down. This very important problem will be considered separately 
in Chapter VIII. 

The next step is the rotation, which is to transform the preli- 
minary factor matrix into a new matrix which, as far as possible, 
meets the requirements for simple structure. Thurstone and his 
followers have generally made this transformation by graphical 
methods. These procedures include subjective elements that 
require experience in the interpretation of graphical representa" 
tions, and are also very tedious, (Thurstone, 1947, pp. 194-410.) 

The many applications of factor analysis to correlation matrice? 
of high rank and the production of highly effective calculator? 
have made the need for objective rotation methods still more 
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es i геп 1 

aperi Even if the five requirements for simple structure would 

BOR ecg for oes overdetermined simple ннат 

‚ they are, however, subjective and ambi 

not summed up in a single i pup e 

ba gle index. Therefore, many atte: 

^ а made to find analytical methods that ue a sue da 

sore ys definition of simple structure. At present, this is 

acto i ich i : 
d едан r analytical theory, which is undergoing rapid 
A ; 

Pow а, of analytical methods is based on “the quartimax 

тр (Saunders, 1953; Pinzka-Saunders, 1954; Carroll 

eos : eyhaus-Wrigley, 1954; and Ferguson, 1954). These 

үн ods, generally speaking, are based оп the idea of maximizing 

Bom a of the fourth power of the rotated factor loadings. Pinzka- 
nders (1954), for example, achieve this, by the function 


X 22 g 
54/0 $ h) (7=1,2...т; k=1,2...m), (78) 


rix of oblique factors. A criterion 


where а,» is a loading in a mat 
mber of high and low 


ee rin: tends to maximize the nu 
Differ ar y avoiding those of medium size (Saunders, 1953, p.5). 
ШЕ improvements of the above mentioned criteria are 
Hirn ү апа tests are going on with them by use of empirical 
Sie ith present it is too early to decide if some of these criteria 
CE aarti at least for solving certain types of problems. 
this tied v indicates, however, that in certain cases criteria of 
ж tend to give rise to general factors also when rotation 

ple structure without а general factor 1s possible. (Neyhaus- 


Wrigley, 1954, p. 90.) 
Thurstone (19542) hasalso tried to develop an 
Which is largely independent of subjective ju 
extension of his single- 


analytical method 
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‘rotational work. This method is an 
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The main principle is that the tests are given weights, which are 
higher, the closer the tests are to the expected hyperplane. 
Thurstone’s analytical method has been improved by Bargmann 
(1953). He has found that the method often gives results that must 
be improved by further rotation (Bargmann, 1953, p. 4). This is 
also the author’s experience of the method. 

Another attempt to solve the rotation problem has been made 
by Tucker, who starts out from the following definition of simple 
Structure (Tucker, 1955, p. 218): 


A simple structure is constituted by the hyperplanes for a set of 7 
linear constellations of dimensionality (7-1). 


Tucker's method is based on the idea of maximizing the 
number of vectors, located within a marginal space of some 
defined and limited width, from each one of the hyperplanes of 
dimensionality (7-1). It is too early yet to express an opinion on 
the effectiveness of Tucker’s new method. He also emphasizes 
that the test battery must be very well-designed if his method 
is to work properly. 

In conclusion: The impression of the various analytical 
methods for rotation to simple structure is that, at present, none 
of the methods results in as good a simple structure as the 
graphical rotation methods give, if these are used skillfully by ап 
experienced investigator. While waiting for improvements O 
the analytical rotation methods, these should be used for finding 


preliminary factor matrices, which then ought to be improved by 
graphical rotation. 


ad 


CHAPTER VII 


THE PROBLEM OF INVARIANCE 


The Concept of Invariance 


It was stated earlier in this monograph that invariance is 
ап important requirement on explanatory factors. Even if one 
factorial analysis were to afford a unique simple structure solu- 
tion, this cannot be taken as sufficient proof that the factors are 
primary in the sense that they have explanatory properties. 
Though the design and the test selection are carefully made, 
Some of the factors may still be artificial products of the very 
Constellation of tests contained in the battery. A single factor 
analysis yields more or less unverified hypotheses as to factors 
Which must be proved invariant through other factor studies 
employing other tests and in respect of other populations, before 
1 can be said that the factors found have the generality required 
ОЁ factors with explanatory properties. 

Compar isons between factor matrices found in regard to differ- 
ent Populations presuppose that the factor pattern is similar, 
11 the sense that approximately the same common factors are 
Présent in the different populations. Even if the same test is given 
to different populations, it does not mean that the same factors 
аге involved in solving this test. Tests which, for a population 
of ten-year-old children, for example, measure, in the majority 
of Cases, factors of intelligence may, in the case of adults, measure 
Perceptual speed. Consequently populations that are to be 
Compared should not be too different, for example, in age, as 
otherwise different factors may be present. . 

hurstone has discussed the requirements for inv: 
Summarized above on page 46. Four cases of invariance E 
Mentioned. In connection with the following discussion p 
Methods of studying invariance, it may be useful to distinguis. 


variance, as 
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between six cases, by dividing each of the two cases of changed 
battery, into cases of partly and of entirely changed battery. The 
six cases refer to when the test battery is the same, or partly 
changed, or entirely changed, and when the populations are 
either the same or different. 

This is evident from the following table. 


TABLE 7. 
Populations 
"Test battery Same Different 

Same (1) numerical invariance (2) configurational in- 

variance Е 
Partly changed (3) numerical invariance (4) configurational in- 

а variance | И 
Entirely changed (5) configurational in- (6) configurational in- 

variance variance 


Each of the six cases gives rise to different problems: 


Case 1.—When the same test battery is used for the same 
population, there should be numerical invariance in all samplings 
of the population. That means that the factor structure found in 
the different samples should be identical within the limits of 
the sampling errors. Studies of this kind of invariance require 2 
knowledge of confidence limits for differences between factor 
loadings calculated for separate samples from the same popula- 
tion. 

Case 2.—1f the same test battery is used for different popula- 
tions, then configurational invariance may be expected to take 
place. This signifies that the change of population affects the 
size of the factor loadings, but in proportion to the changes m 
the variance of the different tests administered to the population- 
This implies that if a test has zero-loading in a certain factor 
in one population, it will have zero-loading in the same factor 
also in another population. This is a necessary condition if the 
same test vectors are to define the same hyperplane. The configu?” 
tion of hyperplanes must, in other words, be the same eve? ! 
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the angles between the normals, which define them, are changed. 
This can be illustrated by the extended vector method (Thur- 
Stone, 1947, pp. 225-258), in cases where there are three com- 
mon factors. If simple structure exists, the heads of the test 
vectors should coincide with the sides of a triangle, for example, 
ABC in Figure 5. 


A, B 


Fig. 5. 


The sides of the triangle АВС correspond to the hyperplanes. 
f another population is investigated by the same test battery 
and is throughout more heterogeneous in these tests, then the 
tests have higher intercorrelations. This will under the assump- 
Чоп of configurational invariance result in the test vectors com- 
ing closer to each other and consequently forming a smaller 
triangle, for example, 4;B,C,. = 
Thurstone has concentrated his attention on determining to 
What extent approximate zero loadings occur in the same places 
în two factor matrices. This is too subjective а method and does 
Rot make use of all the information obtainable from the factor 
Matrices, 
Stricter methods of studying invariance ha 
‘Mong others, by Rasch (1953). He points out t 
8. 


ve been suggested, 
hat the lack of nu- 
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merical agreement between factor matrices is due to the fact that 
the normalizing of test scores and of factor scores has been made 
separately for each population. For, the result of using correla- 
tions is that all tests and factors included in the test battery 
have variances equal to unity. If the loadings in the different 
factor matrices are to be numerically the same for a certain test, 
then test variances and factor variances must be proportional to 
changes between one population and another in respect of the 
actual test variances. Factor analyses must, in such a case, be 
made of covariance matrices, in which variances and covariances 
of a certain test are calculated by the same unit of measurement, 
irrespective of the population for which the test is used. The 
factor loadings thus determined are expected to be of the same 
size irrespective of the population, and may only vary within 
the limits of sampling errors, unless a factual difference exists 
between the factor structure of the populations. The statistical 
testing of the differences will be more complicated than in 
case 1, since the populations are different from each other. А 

Rasch also suggests а method for the simultaneous analysis 
of a number of such covariance matrices, but he has not give? 
any solution which is useful from a practical point of view. 

Case 3.—If the test batteries, used on samples from the same 
population, include tests of which only some are the samo 
the numerical invariance requires that each of the tests that 18 
common to two or more batteries retains its factor pattern. In 
other words, the hyperplanes must have the same position, even 
if other test vectors contribute in determining their position. 
If the hyperplanes are the same, this indicates that the factors 
are not merely accidental products of the particular configura- 
tion of tests іп a certain battery. The problem of sampling error 
is similar to that in case one, but must be concentrated on the 
tests that are common to the batteries. 

Case 4.—Here partly different test batteries are used ОП 
different populations. Case 4 is a combination of cases 2 and " 
above, since both the test battery and the population are changec- 
The configurational invariance signifies, in this case, that the 
same hyperplanes are to be localized, but partly with the help 
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of other test vectors; at the same time also the angles between 
the hyperplanes can be changed. 

It should be possible to carry out a statistical examination of 
differences in factor structure between factor matrices based on 
test batteries which are partly different, i.e. in accordance with 
Cases 3 and 4, by employing those tests which are common to 
the batteries. The procedure ought then to be similar to that used 
for cases 1 and 2 respectively, but is likely to be less certain 
Since, as a rule, a smaller number of tests are included in the 
examination. 

Cases 5 амр 6.—The most convincing form of invariance 
should, in principle, be obtained through the analysis of two 
Completely different test batteries. The risk that an accidental 
Constellation of tests might give rise to artificial factors would 
then be minimal. When one test battery is compared with 
another which is composed of completely different tests there 
are, however, no mutually corresponding loadings in the two 
factor matrices, A comparison must therefore be based either on 
4 subjective impression that a certain factor is the same in both 
cases or on some form of indirect comparison. The latter should, 
In principle, be possible through certain other tests that are 
Correlated with at least some tests from each of the batteries. 


Methods for Finding Invariance 


А number of attempts have been made to devise methods for 
testing invariance. Some of these methods are mainly limited to 
investigations of configurational invariance. Certain of these 
methods are based on the fact that the rows of loadings of a 
factor, which is expected to be the same in two different factor 


Matrices, should have high correlation with each other. Other 
eans of probability models, 


methods c<ample, by m 
compare, for example, by dem 


the profiles of the factor supposed to be invarian 


factor matri 954 
rices (Cattell-Baggaley, 1954). " 
Ore exact tests of invariance must make full use of the numer 


‘cal values of the loadings. For such tests of numerical кк» 
а Common unit of measurement for a test is required, even when 
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the test is given to several populations. The factor analyses then 
have to be based on covariances instead of correlations. The use 
of correlations means that each test, irrespective of its absolute 
variance, is converted into scales with the variance of unity in 
all test batteries. If studies of two populations that have different 
variances in one or more tests, are to be compared, the correlation 
matrices ought to be replaced by covariance matrices, where 
the covariances should adopt values that correspond to the rela- 
tion between the absolute variances of the test in respect of the 
two populations. If, for example, a certain test has the standard 
deviation of 20 points in the first population and 10 in the second, 
then the correlations that include the test should be multiplied 
by the weights which are twice as big for the first population 
as those for the second. Such a change in the standardizing of 
tests can be made also at a later stage of the analysis, viz., in the 
matrices of factor loadings calculated from the correlation 
matrices. This means that the loadings in the test are multi- 
plied by their respective weights. 

This type of criterion of numerical invariance between factor 
solutions in two different populations has been developed by 
Ahmavaara (1954b). 

He takes as his starting point the discussion between Thurstone 
and Thomson on the influence of multivariate selection on 
invariance. Thurstone has given examples showing that if oblique 
factors are accepted and simple structure is used, then the factor 
pattern can be invariant after changes of the variances have 
taken place in some variables, which explains the ensuing changes 
in the intercorrelations. | К 

Ahmavaara (1954а) shows, by employing the Pearson-Aitken s 
selection formulae, that a simple structure solution, according 
to the conditions stated by Thurstone for multivariant selection, 
is invariant. The loadings are Changed in proportion to the 
changes in the standard deviations of the tests and zero loadings 
are consequently unchanged. He also gives a method, based on 
the formulae for correction after selection, for comparing factor 
matrices referring to two different populations with each аш 
(Ahmavaara, 1954b.) First the sizes of variances and covariance 
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indie dies a RUE dt adjusted so that a variable common 
ment. Ашама © : measured by the same units of measure- 
the best bod еа i ^w for a comparison matrix that gives 
ишта Es invariance between the two matrices of 
as the rer an f f. e companion. mariz as of the same order 
te etoscri ^ — and the elements of this matrix express 
їй the other d € as linear combinations of the factors 
then the invari 5 den comparison matrix is a unit matrix 
his method w ira is complete. Ahmavaara has shown how 
compared th orks in several examples. He has, for instance, 
Evrae e = obtained in L. L. Thurstone’s study 
hoi atn А Abilities with those in L. L. and Th. Thurstone’s 
hist thew aser! actorial Studies of Intelligence. Ahmavaara found 
vaara (1957) dun. is good only for some of the factors. Ahma- 
of the method. recently published several other applications 
a pen ‘nae given the problem of invariance investigations 
iS, fusce m than was formerly the case. His approach 
considered r a ES rather than statistical and he has not 
studies mad e problem of sampling errors when comparing factor 
р. 43), e on two different samples (Ahmavaara, 1954b, 
ome E method is applicable when either the same 
from tior e attery which is partly the same, is used for samples 
Cases ME Ls qeu or from different populations. These 
Tucker foci to cases 1-4 in Table 7. . 
matrices di 6) has worked out a method for comparing factor 
ivan te p are based on two completely different test batteries 
Ge 5 $ the same sample. This corresponds, in principle, to 
in Жы Table 7. The two test batteries should be expected 
taped a the same common factors, if the method is to work 
аты ded he correlation matrices for the two batteries 1 and 2, 
d — р by Ru and Rep respectively. In addition, there is 
the ек Ra = Ra, which includes the correlations between 
fion Sim battery 1 and the tests in battery 2. The correla- 
foll matrix that includes all intercorrelations will then be as 
OWs 
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Ry, В, 
R= [ п | (79) 
Ra Roy 


Matrix Rj» is factor analyzed by a specially adjusted principal 
factor solution. No communality estimates are needed since 
communalities are contained only in R,, and Ry», which are 
not included in this part of the analysis. Tucker has worked out 
a test of significance for the residuals in Ry, but he himself is 
doubtful regarding its usefulness. 

The preliminary factor analysis of R,, results in a factor 
matrix, which can be divided into two parts: A, corresponding 
to battery 1, and A, corresponding to battery 2. The factors found 
in both A, and A, are invariant in the two batteries. Each of 
the matrices A, and A, can then be rotated separately to simple 
structure, and the main interest in this type of invariance analysis 
is to study the agreement between the corresponding factors 1n 
the two rotated matrices. This is achieved by a serie of correla- 
tions between corresponding factors in the two rotated solutions. 
Tucker calls these correlations “factor reliability coefficients › 
but until further investigations have been carried out, he 18 
unable to express ап opinion as to what values of this coefficient 
are to be considered as high or low. The coefficient values tend 
to be low if the reliabilities and communalities of the separate 
tests are low. 

This method has many advantages, for instance, it does not 
require communality estimates, it employs an efficient method 
of analysis, and it contains measures of invariance. One dis- 
advantage is, that it does not make full use of the information 
contained in Ry, and Rep, as does an analysis of the whole matrix 
R, according to a conventional method. Therefore, the method 
is more useful when Ry actually includes big loadings in 25 
common factors, which аге found in R,, and R, if they are 


analyzed separately. 
Simultaneous Rotation to Invariant Solutions 


Cattell has considered the invariance problem from pur! 
angle. He is concerned with methods for finding one simP 
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Structure solution, which at the same time, can be applied to 
two or more correlation matrices. He describes this “‘so-called 
principle of parallel proportional profiles" in the following way 
(Cattell & Cattell, 1955, p. 84): 


euge assumption is that, 7f a factor corresponds to some real organic 
POP. raiet from one study to another, it will retain its pattern, simultane- 
idi d or lowering all its loadings according to the magnitude of the 
б s factor under the different experimental conditions of the second 
behave 9 apo factor, a mere mathematical abstraction, would 
дон in this way, within the realm of orthogonal factors. The prin- 
ple suggests that every factor analytic investigation should be carried 
о at least two samples, under conditions differing in the extent to 
the same psychological factors (working as independant, ortho- 

E influences) might be expected to be involved. We could then 
Le a finding the true’ factors by locating the unique rotational 
first rile aris in both studies) in which each factor in the 
ome d B. found to have loadings which are proportional to (or are 
sheild br e function of) those in the second: that is to say, a position 
e discoverable in which the factor in the second study will have 

à pattern which is the same as in the first, but is stepped up or down. 


Я Cattell has devised an analytical method for simultaneous rota- 
ton to simple structure (Cattell & Cattell, 1955). This method has, 
however, some drawbacks, which the authors also have pointed 
Qut. Thus, it is only applicable for rotation to orthogonal solution. 
The basic principle of simultaneous rotation to simple structure 
Seems both important and promising, but e.g. because of the 
last-mentioned reason, we can expect that there will be improve- 
ments in this method. | 

Tucker (1951) has also dealt with the problem of carrying 

Cut simultaneous rotation to invariant solution. He has chiefly 
been interested in devising a method for making the preliminary 
factor matrices what he calls “congruent with each other". This 
Signifies that he tries to rotate the matrices 50 that each of the 
factors will, if possible, have the same content. Unlike Cattell, 

€ does not include restrictions, such as simple structure, 1n the 

Matrices, Tucker considers that these matrices сап, if reasonably 
Congruent, be rotated together to а significant solution, e.g. ac- 
Cording to the principle of simple structure. 


120 


The starting-point for Tucker's method is, to use his symbols, 
the factor matrices F та and Fip, which are two preliminary 
and unrotated matrices with factor loadings, for example, two 
centroid matrices. The subscript j is used to designate the tests 
with unit standard deviations in the separate studies, and m 
and M respectively designate the reference vectors in the two 
investigations. А and B are used to designate the two studies. 

First it is necessary to determine the series of weights by which 
the different tests are multiplied, in order to express each separate 
test in the same unit of measurement in the two studies. These 
weights are included in the two diagonal-matrices D,, and Рв. 
The subscript J designates the tests after adjustment of the 
standard deviations. The next Step comprises the matrix multi- 
plications 

= д 
Ела Dia F ima (80) 
Fius = DjsF JMB 


By this operation the two factor matrices Fma and Fjy; are 
made comparable in respect of units of measurement. 

Tucker then uses the transformation matrices T,,4 and Tura 
which are of such a nature that when 


Ера = Fima Tra and Fre — Fus Tuim (81) 


the factor matrices ЕЁ, and F ув are congruent. Subscript 7 
designates factors in the new solution. The differences between 
these matrices ought to be negligible for tests which overlap 
in the two studies, provided there is invariance. 


Tucker tests this by means of “a coefficient of congruence for 
each factor”: 


Shafi 


зше =, (82) 
WR) Fin) 

The symbols Jira and fj refer to loadings in factor 7 in the 

matrices F;,, and Fp. ф. will be unity when the congruence 15 

complete. The coefficient lacks a test of significance, and Tucker 
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(1951, р. 43) does not wish to state any acceptable minimal value 
of the coefficient of congruence, until adequate experience in 
the application of the method has been gained. 

Tucker points out that mathematical congruence is a necessary, 
but not a sufficient, requirement for identity of the factors be- 
tween two studies. This identity becomes more reliable, the 
larger the number of factors common to the two studies. A 
Psychological analysis of the content of the test should also 
be made, in order to indicate that the factors are not only mathe- 
matically but also psychologically congruent. 

When the congruent factor matrices have been determined, 
they can then be rotated together, for example, according to the 
principle of simple structure, to two new congruent factor 
matrices, These are then invariant in relation to each other. 


Conclusions about the Methods of Finding Invariance 


Differentiation between six cases of invariance has been made; 
Cases which can more or less easily be established on a common 
basis, To rely on subjective impressions that a certain factor is 
the Same in several studies is an unsatisfactory way of investigat- 
ing invariance. А more objective method is to investigate whether 
Configurational invariance exists. Still more objective and efficient 
are the methods which investigate the degree of numerical 
invariance after each test has been adjusted to the same unit of 
Measurement in all studies in which it is included. . 

Ahmavaara, Tucker, and Cattell have all considered this type 
of problem and have contributed towards its solution. At present 
there is no completely satisfactory method which, after the proper 
adaption of units of measurement in the tests, can show “i 
Variance between different factor solutions, and which at the 

; ienificance. The 
Same time is provided with an adequate test of signt ЧЕ 
Methods mentioned above, can no doubt be improved an 


further developed in these and other respects. ENS 
A possible solution would be to design the in g 


basi 
of invariance as tests of hypotheses, е.5. 45 follows. um se | € 
9f the available knowledge regarding both the tests to y: 
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and regarding the earlier factor analyses, hypotheses are formul- 
ated as to the common factor structure and, if possible, also as 
to the numerical values of the loadings. Each test is standardized 
with regard to the same unit of measurement in all the factor 
studies. This implies comparisons between factor matrices based 
upon analyses of co-variance matrices. The hypotheses are tried 
out on different test batteries and populations. The hypotheses 
should then step by step be more exactly formulated and im- 
proved. The problem of the test of significance for such a pro- 
cedure is mentioned in the last section of Chapter VIII. 


Non-factorial Validation 


Numerical invarianceis both an important and necessary condi- 
tion for giving explanatory significance to the factors found but 
it is not entirely sufficient by itself. A requirement stated by 
many investigators is that the factors should also be supported 
by investigations other than factor analytical studies. Thurstone 
(1947, p. 334) formulates this as follows: 


жың ‘ati ill 
The acceptance of primary factors in psychological investigation W! 


depend on the extent to which they are fruitful in non-factorial psycho- 
logical research, 


This implies that we should be able to use the factors, in the 
formulation of testable hypotheses as to relations and regularities, 
in as wide a context as possible. эче 

If, for example, a certain disease, injury or operative uA 
leads to the loss or weakening of a certain factor, but withou 
affecting other factors, it indicates that a functional unity has been 
found. Growth studies, Showing that the development of a 
certain factor is separate from that of other factors, would be 
another type of non-factorial confirmation. il- 

Unfortunately, there are not many such investigations ava! 
able. Thurstone, for example, has made certain attempts along 
these lines in his unpublished Growth Studies of Primary Menta 
Abilities. — 

Eysench (1954, pp. 107—142) has tried to obtain сой 
of political-attitude factors by comparing them with politic 


х= 
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behaviour. He made a factor analysis of correlations between 
questions referring to political attitudes, and found two factors. 
Besides this, he calculated the differences between Conservative 
and Socialist voters, in percentage of positive answers to the 
Same questions. He found a fairly high correlation between the 
loadings of the first factor and the differences in percentage of 
positive answers. This he regarded as evidence that the first 
factor could get a meaningful interpretation as a factor of Radica- 
lism. 

A number of studies of colour vision have been made by em- 
ploying factor analytical methods. As an example of this we may 
mention Ekman's (1954) study, in which paired comparisons 
between colours referring to light of certain wave lengths were 
Studied by an application of factor analysis, called Similarity 
Analysis. Ekman found five colour factors which covered, to a 
large extent, the wave-lengths which Granit, by means of 
Psysiological techniques, found to be connected with different 
receptor groups of colour vision. 

The impression is, however, that on the whole very little has 
been done towards systematic non-factorial validation of factor 


analytical results. 


CHAPTER VIII 


FACTOR ANALYSIS FROM A STATISTICAL POINT 
OF VIEW 


The Need of Statistically Acceptable Methods 


Factor analysis has been developed mainly by psychologists 
as a mathematical technique for the treatment of test results and 
other measures of individual differences. Statisticians have 
worked out solutions for only some of the statistical problems 
involved, but are now concerning themselves more and more 
with the whole field. The tendency has often been for psycholo- 
gists themselves to develop factorial methods, which are inef- 
ficient from a statistical viewpoint, for converting a correlation 
matrix, with the exception of negligible residual correlations into 
a matrix of factor loadings with low rank. It is only subsequently 
that they have tried to deduce the sampling distributions, and 
to determine confidence intervals for the residual correlations, 
in order to make it possible to test whether these really are 
negligible, and to decide when to stop factoring the correlation 
matrix. Because the psychologists have not made full use of 
current statistical theory, the confidence limits for the residual 
correlations are very often given an unsatisfactory basis. Many 
of these attempts have been characterized by their authors as 
approximations only to be used for want of anything better. 
Burt (1952), among others, has discussed these methods thor- 
oughly. 

These conditions have led many specialists in statistical theory 
to be both hesitative and critical in respect of factor analysis 
and have also caused them to endeavour to persuade the psycho- 
logists to formulate their problems differently (Hotelling, 1942). 

Some statisticians have only recently accepted the distinction 
in psychological research between component and factor analysts, 
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Which is made in Chapter I. One reason for this is probably 
that for a long time only one method existed, which is satisfactory 
from a statistical viewpoint, and which is useful in component 
analysis, viz. the principal component solution. Any counter- 
part to this, which is useful in factor analysis, has been lacking 
for a very long time. Efforts have been made, however, to use 
principal component or related methods also for problems of a 
factor-analytical nature. 

Before discussing these approaches, it might be well to repeat 
the distinction between component analysis and factor analysis, 
Which was already made in Chapter I. 

In component analysis two cases may be distinguished. 


(1a) а= X as Fy (31,2... 21 {=1,2 iN: ей. 0 
i (83) 


Each test score is here considered as a weighted sum of 4 
Components. These in their turn are transformations of the 
п tests, and make possible the exact representation of the test 
Scores. The component loadings are found in matrix 4, which 
follows the relation R,=A®A’. In psychological applications, 
q will probably be equal throughout to the л tests of the battery, 
Since each test will, in practice, partly measure something specific. 


a А аң = > ак Fry t Ge &jk- (84) 
Kel 


d by errors of measure- 


In this case, each test score is impaire 
ase 1a, 


ment. The component loadings form a matrix Ay As in Е =. 
9 in this matrix is often equal to л. The л loadings for : ә ч 
factors constitute the diagonal matrix A,. In case 1b the following 


Matrix relation is valid. 


R, =A, D A54 4. (85) 

Also in factor analysis two cases can be distinguished. They 
аге called 2a and 2b. 

(86) 
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This model corresponds to case 1a in component analysis 
but also includes one specific factor in each test, the loadings of 
which are found in the matrix А;. The different common factor 
loadings a; correspond to common factors in the matrix Am 
The introduction of specific factors will, as experience shows, 
make it possible, as a rule, to reduce the number of common 
factors m to considerably below the number of tests л. 

'The relation between the correlation matrix R, and the factor 
loading matrices is 


R,- A, d А„+ 4%. (87) 
In addition R=A,¢ An. (88) 


The matrix R is a correlation matrix with communalities in the 
diagonal. 


m 


(2b) зп =, Ў аъ Fair Qs Spt ауен (89) 


In this case the test scores also contain errors of measurement i 
the loadings of which form the matrix A,. Then the relation 1$ 
given by 

R,= Anp Al, + AB + AB (20) 
In addition à R-4,4 A;. (n 


The inclusion of specific and error factors in the model, which 
together are called unique factors, makes it possible to reduce 
the number of common factors m in the matrix А, so that 
they are considerably less than the number of tests л. 

The main distinction between factor analysis and component 
analysis is that the former includes specific factors. In psycholog- 
ical applications this facilitates the reduction of the number of 
common factors, i.e. the rank of matrix 4,, and also makes i 
easier to obtain invariant solutions, i 
The distinction between component analysis and factor analys!$ 

conditioned by the nature of the problems in behavior? 
ence. Component analysis as applicable to case 1b, where 
an error term is included in the model, and both the cases Е 
factor analysis can however, be treated by the same statistica 
method (Whittle, 1952). This will be shown below. 
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The ideal solution of the factor analytical problem, from a 
Statistical viewpoint, would be to have an efficient method which, 
without rotation, leads directly to a psychologically significant 
solution, i.e. according to the principle of simple structure, and, 
Which, at the same time, permits the development of fully ac- 
ceptable confidence limits for the residuals and the different 
loadings. 


Maximum Likelihood Solutions and Least Squares Solutions 


. In their efforts to develop acceptable factorial methods stat- 
isticians have used approaches based either on maximum likeli- 
hood- or on principal component solutions. In 1940 Lawley 
Published a maximum likelihood method of factor analysis in- 
Cluding a test of significance for residual correlations. The 
factorial method called Lawley’s method I, is based upon the 
assumption that all test scores and factor scores are statistical 
by nature and have normal distribution. The test of significance 
18 useful for large samples, and by means al Bartlett sitaan) 
adjustment it can also be used for medium-sized samples. 
The method has as its starting point a hypothesis as to de 
number of common factors, i.e. the rank of the matrix R. On 
the basis of this hypothesis an orthogonal factor matrix is esti- 
mated, corresponding to the population correlation matrix for 
Which the likelihood of occurrence of the observed апе 
matrix іп the course of random sampling isa maximum. With the 
aid of the test of significance it is determined whether the residual 
matrix, when the hypothesized number of factors are abstracted, 
can be considered to be dependent on sampling En or 
Dot. If not, then the analysis is repeated on the basis of anot үг 
Ypothesis as to rank. Estimates of communalities e а Fi 
made in the analysis, but will be obtained. as an end cd 
The method has been tried out on ie perm the 


Populations whose factor structure is know 
ү : used 
"test scores" are normally distributed. Henrysson (1950) 


ini and 12 
а System of 9 variables containing one common «oy e. 
Samples each comprising 200 observations were taken. y 
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and Swanson (1954) used a system of 7 variables containing two 
common factors and took 8 samples each consisting of 50 obser- 
vations. They also analyzed the total sample of 400 observations. 
The results of these studies seem to indicate that both the method 
and the test of significance work well even for samples of only 
50 observations. 

Lawley’s method I has certain drawbacks. It is built on the 
assumption of random distribution of factor scores and test 
scores. Andersson and Rubin (1956, pp. 130 and 145) have shown, 
however, that for large samples the method can also be used 
when this assumption is not fulfilled; to what extent it must be 
satisfied is not yet known. 

Whittle (1952) has developed solutions using least squares 
methods and has shown that this leads to principal component 
solutions. By this approach he avoids Lawley’s assumption, and 
regards both loadings and factor values as parameters which 
have to be estimated. Young (1941) has dealt with the same prob- 
lem in the simple case of rank one. Whittle discusses his method 
mainly in connection with the model 


а 
2j = È аъ Fert ye ё. (2) 


This corresponds to case 1b of component analysis. In the article 
in which he discusses his method, he treats a model which only 
includes, besides the “common factors”, what he calls “residuals 
or “accuracies” (Whittle, 1952, pp. 223-225). А 

Whittle, however, points out that his special form of principal 
component solution is applicable even to the factor analytical 
models which include specific factors, i.e. 


m 3 
Hy - Да Frit as S, nd 

m 4 

or B= 2, p Frit dys Sjit a5, ёд. 0 


In the first of these formulas what Whittle calls “residual errors’ 


i + 
are represented by а, S}; and in the second formula by ау; Эл 
ау, ёз. 


v 
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Another wa i a 
" y of expressing th ittle’ 
15 valid for the md. ssing this is to say that Whittle’s approach 


B= > ay Fata Оң. (95) 


Thi 4 

ин p IM a unique factor, which can be composed in 
рейс ion may consist of only an error factor, or of only a 
P sss E or of both an error and a specific factor. The last 
2b E to factor analytical models, to cases 2a and 
Es Seanad: ove. In Whittle’s approach all these three cases 

er oan in one statistical method. 

f Bier decem factors are included in the model the number 
on ean » ings will be very large and the problem of estima- 
iat more difficult (Holzinger-Harman, 1941, p. 265). 
With the E yere epe: in the discussions in connection 
Sali Буйрек. vi is ^ EN in Uppsala in 1953 (Upp- 


Dr. Ў : 
sary, Bp aated if the introduction of a specific factor was neces- 
Кураны " a it did not imply a surrender of the essential factor 
Factors E ou da battery of 2000 tests really need 2000 or more 
ен спас It is possible that the apparent occurrence 
рай. ue to the fact. that the number of real factors is large in 
n with the usual size of a test battery. 
c factors are 
p. 63) 
y the 


That Whittle’ ; 
i ОЕ s method is useful also when specifi 
wis E in the model is pointed out even by Wold (1953, 
à evertheless also mentions the difficulties incurred b 


inclusi eel. m 
sions of specifics in the model: 


Suc i 
h specific factors do not enter Whittle's theory, but it is clear 


that j 

if п я z A К 

(hs ie wish to do so we can interpret his residual as a specific factor 
ernatively as the sum of a specific factor and a measurement 


error 2 З 
). This change іп the interpretation does not affect the formal 
other question, however, 


dey 
tenes of Whittle’s method. It is quite an 
Eom ев the introduction of specific factors is at all logical or appropriate 
C PN viewpoint of the fundamental idea of factor analysis, that of 
їс ining a set of test scores in terms of a smaller number of psycholog- 
al factors. 
10 Ву employing Whittle's methods one can estimate both factor 
adings and factor scores. The only general assumption required 
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is the ordinary one, ie. that all factors, including even the 
unique ones are uncorrelated with each other. Whittle's object 
is to solve A,, in the equation 


R,-A,4, +А, (96) 


but he is unable to do so for the general case. However, he 
considers two special cases of interest, viz. the following (Whittle, 
1952, p. 229): 


We can distinguish, however, at least two special cases which deserve 
further attention. The first is that in which the relative accuracies of 
the different measurements are known. The measurements may thus be 
standardised so that all residuals have the same variance, and we are 
back in the case of the previous section (except that we have now 
allowed for variation in mean value). The second case is that in which 
the error variance of a measurement may be assumed proportional to 
its population variance...” 


АЕ ion 
In the first case, called *the additive case", the assumptio 
is made that all unique variances are alike, i.e. 


@jy=const. (7=1,2,... n). (97) 


Since іп most actual cases the unique proportions of the he 
variances are different, each test variance must be standardize t 
зо that all unique variances become equal. This presupposes so 
instead of the correlation matrix a covariance matrix is considere 
such as the following 


Ry! R, Rọ. eu 

This relation gives the equation system 
9 
| y! R, Rot - 4 1|- 0. 099 


The latent vectors according to this equation system will 
after adjustment, correspond to factor loadings. | 

This solution assumes that the test variances are standardize " 
in such a way that the unique variances are equal. Their ee 
need not be known, however. It is sufficient to know owe 
relative size in relation to each other, so that they can be m 
equal by changing the sizes of the test variances. 
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in б yar ae : this is apparent from the coordinate system 
wi, tn sits е x-axis corresponds to the total test vari- 
imu et Ji « n ardized, and the y-axis corresponds to the 
ne d rekon variance. In the additive case the unique 
=ч ee e equal in size regardless of the size of the 
абд е пе the unique variances should form 
Whittle itis M nh same distance from the x-axis. According to 
ЖЕБЕ ia d s to assume that the line is horizontal. The 
boue istance a from the x-axis does not need to be 


Unique 
variance 


Additive case 


Multiplicative case 


Total test variance 


Fig. 6. 


it is assumed 


In iplicati 
the latter case, called “the multiplicative case", 
ec- 


that i 

a i i i 

de ll unique variances are in the same proportion to the resp 
€ test variances, i.e. 


dij-cons.$ (j= 1,2, +”). (100) 


6. The unique variances are 


This ; ? А " 
his is also illustrated in Fig. 
e test variances. 


c ud : 
лн in direct proportion to the chan 
e relation corresponds to the tangen 


Si i 
"i of which does not need to be known. 
is sufficient to assume that the line passes through the origin. 


The assumption of constant relation means, in this case, that 
Unique variances should be of equal size ifa correlation matrix 15 
analyzed, because in such a matrix, all test variances are standard- 


ge in th 
t of the angle p, the 
According to Whittle, 
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ized to the same size. The size of the unique variances need not, 
as previously mentioned, be known. It is sufficient to assume that 
the proportion of the unique variance is similar for all tests. The 
equation system in the multiplicative case will be 


| R, - i1|-0. (101) 


The assumption in the multiplicative case, that all unique 
variances are proportional to the respective total variances, will 
very seldom be satisfied in psychological and similar applica- 
tions. The additive case will therefore be of greater interest, at 
least as regards the behavioural sciences. 

Wold (1953) has tried both the additive and the multiplicative 
cases on artificial samples, when the scores of the two common 
factors were sampled from the positive half of a normal distribu- 
tion, and the residuals were based on independent random 
samples from a normal distribution. 'The common factors were 
taken from a truncated distribution in order to make it possible 
to investigate whether the method works even when the distribu- 
tion of the common factors diverges considerably from the normal 
distribution. There were five “tests” and the size of the sample 
was 50. Wold also made an experiment with the additive case 
on an artificial example consisting of two tests with one common 
factor and the size of the sample was 50. He found that Whittle's 
procedures worked quite satisfactorily in all three examples 
and gave acceptable figures both for loadings and for individual 
factor values. 

Whittle's approach has the advantage over Lawley's that tos 
solutions for loadings and factor scores do not require any stipula- 
tions regarding the form of the distribution of the factor scores 
or test scores. On the other hand, the two different assumptions 
as to the size of the unique variance in relation to the test Vat!” 
ance involve limitations that Lawley does not need to make. 
The fact that it is not necessary to know their numerical values, 
and that we need only assume that they are equal after suitable 
standardization of the different test variances, is, however a 
relief. It would be of value, if investigations were made in the 
future to ascertain how strictly the requirement, that the unique 
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variances m NS 
principal ien а in size, need be satisfied in order that the 
After css Jede te may work satisfactorily. 
не А ittle's approach, confidence intervals can 
"isum ipee ae and factor scores and hypotheses as to 
is based on the A cone Bus tested Tap Pariter tise, o 
of a covariance maki v ам developed by Wilks for the roots 
the wipe fetora ix ( аген, 1950 and 1951). It assumes that 
means that the si nif айыра wifi normak Она d 
analysis, requires os cance test, contrary to the method of 
mally distributed Serle errors and specific factors are nor- 
it would be зав а jon assumed for the specific factors, then 
зом он па y to consider also the common factors as 
Чу исо шша distribution. The fact is, that there is no 
neris x Le limit or factual difference between the 
Common ain s parts of the variance in a certain variable. 
the test is anal an a test included in a certain battery can, if 
— rel i. E another battery, fully or partly become 
Ысы haw ai contrariwise. However, it has not yet been 
satisfies! 1a orde ox the requirement of normality must be 
factorily. Tt sh doy the test of significance to function satis- 
20:5lity às me also be pointed out that the assumption of 
б-а Hena vigi only for the test of significance but not 
eser € med gei solution itself. 

oni or А dde that Bartlett and Lawley in discussions with 
conducting an е eame were critical nt the possibility of 
the Siria E ysis according to Whittle’s methods, under 

at both the common factor loadings and factor 


Scores 

a: . . 

stern те parameters to be estimated, since the number of para- 

increases in proportion to the number of observations. 
int in a critical review 


K 
endall (1954, p. 483) makes the same po! 


of 
the Uppsala Symposium. 
because the number of parameters 


ations. 


his criticism that: There are two 
d; the first refers to the estima- 
gs, and the second refers to 


TI i i 
Edere. isa substantial difficulty here 
estimate exceeds the number of observ: 


Whittle and Wold reply to t 
prm of estimation involve 
on of n х m common factor Joadin 
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the estimation of N x m common factor scores. The errors of 
estimation of the loadings approach zero when the size of the 
sample, N, approaches infinity, but this is not true for the errors 
in estimating the factor scores. On the other hand, if the number 
of tests, 2, increases, the errors in estimating the loadings will 
not approach zero, while the errors in estimating factor scores 
will approach zero. . 

As mentioned in Chapter I, in behavioral science it is con- 
sidered more important at present to obtain estimates of the 
loadings than of the factor scores. It will be possible to obtain 
acceptable estimates of the loadings by Whittle's methods, especi- 
ally if the size of the sample, N, is large and the number of com- 
mon factors, m, is small. . 

Lawley (1953) presented his Method III, which was partly x 
fluenced by Whittle's investigations, at the Uppsala Symposium. 
With this method he obtains estimates of the factor loadings, and 
makes the same two assumptions as to the unique variances 2$ 
Whittle does. The method works, however, with the assumption 
that the factor scores are normally distributed statistical variates. 
The method is also provided with a test of significance for de 
samples. Lawley has tested his new method on an artigo 
sample of 400 observations, 7 tests and 2 common factors, an! 
has obtained good results (Lawley, 1955). " 

Whittle’s methods and Lawley's method III have certain 
drawbacks in common. In the additive case the unique factor 
loadings are assumed to be equal after suitable adaptation of the 
Size of the different test variances. Some knowledge of the [os 
of the unique variances is necessary for this adaptation. Inthe 
multiplicative case the residuals must form a constant propor” 
tion of the test variances, a condition which will seldom apply !2 
practice. It is not yet clear how sensitive the methods are 10 
biased or inefficient estimates of the unique variances or of their 
relative size. 


^ š z j "NS it- 
Pending further investigations of the characteristics of Whi 


n 
* Lawley (1942) has also devised a Method IL, which, however, has bee 


shown not to converge towards an acceptable solution (Bartlett, 1950, PP: 
80-81, and Whittle, 1952, p. 224). 
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› 
седа, Lawley's method I would seem at present to be 
EIC eene eem 
airean poster 1 princip. es, since i does not 
Ӯ y wledge of the size of the unique variances. It 

3 —€— however, in principle, normal distribution of the 
Psi ven test scores. As has already been pointed out, 
Arr, jen n icu and Rubin have indicated that the method 
tións ате y we for large samples even when these assump- 

not satisfied. 
aioe computers are available, Lawley’s method I 
spin нар deal of work, especially as it seems to be a neces- 
аа а the initial set of trial values for the factor 
is to pR i- ould lie very close to the solution, if the convergence 
anA ids properly. (Lord, 1956, p. 39.) This difficulty can be 
valts: dn agri extent by the use of centroid loadings as trial 
indus is > iteration. The centroid method is often able to 
Сй: adings that closely correspond to maximum likelihood 

- (Lawley, 1955). 
-— to investigate this correspondence Lawley has com- 
ipla 15 method III with a centroid method on large samples 
ъс АЫ к with normal distribution. He selected his 
175) I instead of I, for the following reasons (op. cit., р. 
Eos ен eee for the assumption will, however, be that the 
in which kie га 15 to be compared with a vera e OUS 
teasonably be E residual variances are каас to s es t шыр, 
Váriancsg м ^F ida that any inaccuracies in t Е wa id t pee cm 
AGE serio TM ave much the same effect on both methods ап 

sly invalidate the comparison. 
Lawley expressed the opinion 
e whole, also applic- 
gation, Lawley 
on. It has the 


In discussions with the author, 
that the results of the comparisons are, on th 
able to method I. In order to facilitate the investi 
describes a modified form of the centroid soluti 
following characteristics (op. cit., р. 176): 

(i) the residual variances are assumed to be known, 
(ii) the sign changes are determined in advance, 
(iii) the process is applied to the covariance matrix. 
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Lawley considers that the results of his investigations show that 
the centroid method is more efficient than is generally believed. 
He concludes his analysis by saying (op. cit., p. 189): 


To sum up, we may say that the centroid method, at least in its modi- 
fied form, has in general fairly high efficiency and has the advantage 
of being relatively simple to perform, as compared with the maximum 
likelihood method. It has the disadvantage that the 7? test of signifi- 
cance is not so convenient and exists as yet only in a large-sample form. 


The twelve correlation matrices, based on twelve artificial 
samples which are mentioned in connection with the discussion 
about communalities, make it possible to compare the centroid 
method with Lawley’s method No. I in the case where the 
expected rank is equal to one. In connection with another in- 
vestigation (Henrysson, 1950), the factor loadings were calcu- 
lated by Lawley’s method I for covariance matrices based upon 
the same data as those relating to the twelve correlation matrices. 
As Lawley (1940, pp. 69-70) has shown theoretically and Burt 
(1950b, pp. 161-163) empirically, the loadings according tO 
Lawley’s method I in both types of matrices stand in a simple 
relation to each other. Each loading in the correlation-matrix Сап 
be calculated from the loadings of the covariance matrix by 
dividing them by the square root of the variance of the test 
in question. 

The centroid loadings for the twelve correlation matrices have 
been calculated and are given in the appendix. The commu- 
nalities were calculated through iteration of the centroid solution 
until the communalities were equal to the squares of the centroid 
loadings. 

The loadings according to the centroid method and Lawley’s 
method No. I are given under each correlation matrix in the 
appendix. The loadings according to both methods are very close 
to each other. Out of the 12 x 9 = 108 pairs of loadings it is only 
21, which differ by more than 0.002. The conclusion reached 15, 
that at least in this simple case, both methods give loadings 
which are very similar. 
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Tests of Significance for Factor Loadings 


In factor analysis there are two related problems regarding 
tests of significance, both of which assume a knowledge of 
confidence limits (Burt, 1952, pp. 131-133). The first problem is 
When to stop factoring. This problem assumes that there are 
confidence limits which make it possible to test whether the 
differences between the matrix А.ф А, and the correlation 
matrix R are significant or not. Several methods for accom- 
plishing this have already been discussed in this chapter. 

The second problem is concerned with confidence limits for 
different loadings in matrix 4,. Knowledge of such limits 
Should, for example, make it possible to test whether a certain 
estimated loading deviates from zero or not. 

Psychologists have often tried to establish confidence limits 
for factor loadings by very rough methods. Certain rather arbi- 
trary rules of thumb are used for the investigation of large 
Samples, generally without adequate reasons. It is customary, for 
example, to consider loadings outside the interval of 0.30 
10—0.30 as significant, while loadings in the interval of 0.10 
t0—-0.10 are considered to be approximately zero. 

A number of attempts have been made to find formulae for 
the standard error of a single factor loading, including zero load- 
ings. These formulae are discussed by Burt (1952, p. 127-133). 

The only acceptable tests of significance for loadings would 
Seem to be those introduced by Lawley (1949) and Whittle 
(1952). These tests are, however, based on the same assumptions 
as those in respect of their methods of analysis. Factor matrices 
calculated by these methods must, however, be rotated to give 
Meaningful solutions in behavioral science. à 

A procedure, which accepts as significant loadings outside cer- 
tain limits around zero, and which considers as Zero i 
those which lie within a certain interval around zero, 15 UN- 
Satisfactory from a statistical viewpoint, even if the ds 
based on a knowledge of the sampling distribution. When dealing 
With a large number of loadings, this procedu 
1n many errors of acceptance and of rejection. 


re can result both 
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It would be more satisfactory if the testing of significance 
could be carried out in connection with hypotheses as to the 
whole factor structure. The investigation of significance could 
be conducted, for example, by studying the sum of the squares 
of the differences between empirical loadings and loadings in 
accordance with an hypothesis. Such hypotheses could be based 
on a certain theory or on earlier factor analyses. In the later case, 
the comparison is at the same time a study of invariance, as the 
basis for the hypothesis is a factor matrix obtained in another 
investigation. This matrix is compared with the factor matrix in 
a new investigation with the same, or with partly the same battery 
of tests. 

It is also conceivable to test less specified hypotheses, for 
example, that simple structure exists in regard to a certain 
test battery. In this case, the hypothesis implies a number of 
zero loadings distributed according to the principle of simple 
structure. The testing of such a hypothesis, which is difficult to 
formalize, particularly in the oblique case, could, for example; 
consist in minimizing a sum of the squares of the deviations from 
а factor matrix containing suitably placed zero loadings. A solu- 
tion could perhaps be reached, if Lawley's or Whittle's methods 
were further developed so that certain restrictions, for example; 
the presence of zero loadings, could be introduced and tested. 
At present, there is no general method available for testing this 
type of hypotheses as to factor structure, although attempts 
have been made by Howe (1955, pp. 104-114) and Anderson& 
Rubin (1956, pp. 137-138). They discuss restrictions on max 
mum likelihood solutions, which imply zero loadings according 
to modifications of the principles of simple structure. By this 07 
similar methods the two problems of confidence limits could be 
merged into one, viz. the problem of finding confidence limits 
for the differences in 


R-An$ An ш) 


with certain restrictions imposed on A,,. 


SUMMARY 


dus үш Жы: to an investigation of what type 
арыш inn : Е avioral sciences may be solved by factor 
9з аа} н, ci i studies ought to be designed and carried 
Ilia p iem е the technical problems in connection with 
а qs ns ae ~ be solved. The discussion is con- 
ебу. onal form of factor analysis, the so-called 
e a short review of some important aspects of 
agen 4 de рл of factor analysis. The basic equations 
гурчй p e оаа are given and discussed. 
Stillen, Tide at the general equations do not give unique 
lathean aE restrictions on the solution must be made. 
ona р ert e assumptions, as to the nature of the data, are 

ered, which have to be satisfied if factor analysis is to give 


meaningful results. 
Neu surveys the most important schoc 
Mi ow 2 he models they use. The limiting restri 
The amen SUR employing different models are given prominence. 
ledio ifferences between the various schools refer to the 
gud Fs es governing the choice of the pattern of zero loadings, 
what extent negative loadings should be accepted. It is 
gen agreed that factor analysis is a useful descriptive tool; 

ut some leading scientists in the field, mainly Spearman an 

Thurstone also wish, under certain conditions, to assign to the 
factors explanatory properties which refer to the nature of the 
underlying structure. 
Chapter III discusses three main types of application of factor 
analysis. The first refers to the problem of finding the number of 
dimensions. It is pointed out that factor analysis has made an 
important contribution to psychological theory and application 
by stressing, and giving more exact meaning to the concept of 


hools of factor 
ctions which 
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dimensionality. This refers especially to the field of scaling. In 
this connection, the concept of homogeneity is discussed and 
clarified. The second type of application consists in employing 
factor analysis for finding condensed and simple descriptions of 
a large number of observed variables and for describing the 
relations between them. The choice of the model, in this case, is 
a matter of convenience, where certain general requirements in 
respect of the descriptive models or of the special field of applica- 
tion are decisive. How factor analysis can help in finding batteries 
consisting of valid and low-correlated tests which are useful for 
predictive purposes is also discussed. This is applied especially 
to the field of differential prediction; and it is shown that here 
the use of factor tests is an attractive solution from a theoretical 
point of view. Though the problems involved are many, the 
practical results obtained are few. The third type of application 
comprises the use of factor analysis for finding factors with 
explanatory properties. 

In Chapter IV the distinction between descriptive and ex- 
planatory factor analysis is considered. For this purpose, results 
are applied which are obtained in connection with the theory of 
science and the discussion of intervening variables and theoret- 
ical constructs. The main finding of this investigation is, that it 
seems possible to distinguish between descriptive and explana- 
tory factor analysis, even if this is difficult to apply in practice. 
Descriptive factor analysis is, in principle, only a deductive 
procedure Which gives operationally defined factors related оч 
certain set of data. Explanatory factor analysis is an inferential 
procedure, the object of which is to find generalized factors with 
surplus content referring to regularities and relations outside 
an actual factor study. Invariance is an important requirement 
in respect of explanatory factors. 

Chapter V deals with the problem of how explanatory factor 
analysis ought to be designed and carried out. This can be done 
either in a hypothesis-testing or in a hypothesis-generating man- 
ner. 

Chapter VI contains a discussion of factor analysis conducted 
according to the principles of simple structure, and of some 0 
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the technical problems involved. The main principle is to find a 
set of hyperplanes of lower dimensionality than the rank of the 
correlation matrix. Moreover, it is shown that the principles of 
Simple structure, as at present stated, are not always sufficient 
to afford unique solutions, especially in the case of rotation to an 
oblique solution. Different analytical methods for rotation are 
discussed, and it is shown that none of them as at present 
constituted, is fully satisfactory, and needs to be supplemented 
by graphical rotation. Requirements in respect of the tests in the 
battery are examined, and it is shown how differences in the 
reliability of the tests may interfere with the choice of hyper- 
planes during the rotational procedure. The problem of estimat- 
ing communalities is treated. For all applicable factorial methods, 
except Lawley's method I, it is necessary to make, in advance, 
estimates of, or certain assumptions as to communalities. In this 
connection, it is demonstrated both algebraically and by experi- 
ments on artificial samples that a principal component method 
without communalities, as suggested by Thurstone, is useful 
only in the case of rank one. 

Chapter VII deals with the problem of obtaining in 
between different factor solutions, after changes have bee 
in the composition of the test battery or in the population. Me- 
thods based on the principles of simple structure can in many 
Cases give invariant solutions. Subjective inspection and other 
inefficient methods of studying invariance are criticized. Different 
methods for studying numerical invariance are discussed. These 
methods require that the same units of measurement are used 
for a test analyzed in different factor studies. This often implies 
that co-variances instead of correlations should be analyzed. It 
is pointed out that an attractive way of studying numerical in- 
variance would be to consider factor patterns found in earlier 
factor studies as hypotheses which require to ре sested — 
factor analyses. Factor analytical studies of invariance ought 


to be supplemented by non-factorial validation of the factors. 
sidered as a technique 


In Chapter VIII factor analysis is con: v: 
of statistical inference. Many of the methods used by psycho'o 
gists are unsatisfactory from a statistica 
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1 point of view and lack 
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acceptable tests of significance. Methods developed by statisti- 
cians are generally based on maximum-likelihood or on principal- 
component solutions. "These methods are discussed mainly 
from the point of view of the underlying assumptions and of 
their applicability to the behavioral sciences. It is pointed out 
that these methods should be adjusted so as to give solutions 
which are meaningful for data in the behavioral sciences. The 
available statistical methods for testing hypotheses as to in- 
variance are still not satisfactory. Current statistical methods 
for testing significance should be adjusted so as to make it 
possible to test hypotheses as to explanatory factors and their 
invariance. 


APPENDIX 


Artificial correlation matrices and loadings found by four 
different methods 


Row 1 under each correlation matrix contains loadings by the 
principal component method with communalities and row 2 
without communalities. Row 3 contains loadings by Lawley's 
method I and row 4 loadings by the centroid method. The chi- 
square value under row 4 is calculated from the solution by 


Lawley's method I. 


i .471 -546 -499 
2 471 d An A34 458 474 I9 459) RE 
3 -546 .459 .567  .444 -525 .481 .592 B 
4 4499 -434 -567 i528 .12 491 493 505 
5 483 455 444 528 490 475 473 552 
6 „521 .474 .525 +512 «490 .541 «5I p 
7 по 419 481 49r 475 54 45 e 
2 1530 452 .592 493 473 .518 .439 М 
9 .537 432 -504 .500 «552 .508 .542 - 

PC 

with А? 729 .628 .734 714 .688 -726 89 T da 
тое p 720 629 .734 -714 ГВ 1% ie po pee 
entra A Ча ae p jy 689 gras 2728 


Centroid .729 .629 -733 


%2=21.8 
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SAMPLE П 
Variable x 2 3 4 5 6 7 8 9 
I -644 .487 601 .616 .sss -548 .637 -527 
2 -644 544 .583 .622 .485 .589 .546 .573 
3 487  .544 488 .599 .58o0 .s21 .566 .540 
4 бот .583 .488 -604 .529 .619 .585 .562 
5 :616 .622 .599 .бо4 559 .558 .619 .бїї 
6 +555 .485 .580 .529 .559 -554  .607 .517 
7 -548 589 .521 .619 .558 .554 .548 -567 
8 4637 .546 .566 .585 .6:19 .607 .548 -583 
9 "527 .573 .540 .562 „біг .517 .567 .583 
PG 
with A? 768 .761 .711 .759 .799 723 -745 .781 .741 
without Л? 768 .76: угт 759 .799 .723 .745 -781 -741 
Lawley 771 763 .711 .760 .799 .721 .743 .779 .74! 
Centroid 767 761 712 -758 .799 .723 .745 .781 .74ї 
2 = 38.6 
Samre III 
Variable I 2 3 4 5 6 7 8 9 
1 510 .540 510 .549 .$21 .534 -535 196 
2 510 +528 .426 442 .452 .414 531 +419 
3 "540 .528 :582  .532 .529 .426 -526 An 
4 510 .426 .582 4470 .552 .464 -505 +4 H 
5 1549. 442 .532 дуо -536 .428 -503 ju 
6 521 .452 .529 .552 .536 .464 -520 ko 
7 "534 414 .426 464 428 464 404 4 i 
8 "535 .531 .526 .505 .503 .520 .404 5% 
9 400 .419 .495 .489 .478 -535 -482 .544 
РС 6 
with hè 4747 -656 .743 ут .699 .732 .634 725 o 
without А2 747 -655 .743 .711 699  .733 -633 -724 “bee 
Lawley 744 657 .743 .713 .уот .732 .633 .724 "606 
Centroid -749 .655 .743 -710 698 .733 .633 -724 - 
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ЅАМРІЕ IV 
Variable I 2 3 4 5 6 7 8 9 
I 521 .478 .516 .513 440 -523 494 462 
= -521 483 -483 -530 -439 .502 -499 .503 
3 478 .483 .523 .486 .453 -543 476 -541 
4 516 .483 -523 497 -532 .529 .544 -476 
5 .513 «530 .486 .497 .512 .457 +559 -503 
6 .440 .439 -453 -532 .512 65 469 -444 
7 .523 .502 .543 .529 457 465 482 .490 
8 1494 499 -476 .544 -559 469 482 505 
9 462 .503 .541 .476 .503 444 490 505 
PC 
with Аз уоо .7o2 .7o7 .730 -721 66: .708 .716 -695 
without Ла лоо .7o2 .707 -730 721 1662 .7o08 .716 -695 
Lawley лог .7o2 .706 .729 .720 1663 .708 717 695 
Centroid ‘Joo .7o2 .707 .731 722 66 .708 .716 -695 
43-19.2 
SaMPLE V 


.543 -388 .398 -452 


I 
® . „бід 513 587 +577 ` 
3 388 «512 2 p „512 -503 +590 439 R^ 
4 .398 .513 -516 дот -503 E de E 
5 452 .587 -512 407 .516 -506 n E 
Э .489 .577 -503 503 :516 А + S. oc 
7 .488 .578 -500 -515 .506 .461 "e pei 
8 .473 482 -435 .426 422 E^ p 333 
9 490 .542 492 434 85 484 - 
= 6 .710 
with А? 6661 .787 .687 -657 .695 pei 25 M = 
without Л? .661 -786 -687 .657 +095 i TU 66 -707 
Lawley 1663 .788 -685 638 m T 25 Ж 


Centroid 1669 786 -687 


y= 28.1 
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Samp_e VI 
Variable I 2 3 4 5 6 7 8 9 
І 422 .501 .372 .423 .456 .464 .413 .396 
2 422 540 .477 .481 .522 .511 .431 .459 
3 .SOI .540 517 .490 .561 .575 -476 .524 
4 372. -477 -517 487.538 .558 .468 -518 
5 423 .481 .490 .487 .490 .441 .497 -468 
6 456 .522 .56r .538 .490 .577 -456 -575 
эў 464 .5II .575 .558 .441 .577 -450 Be 
8 413 431. .476 .468 .497 .456 .450 “т 
9 396 459 .524 .518 .468 .575 .531 4478 
РС 
with А2 -607 .687 .756 .706 „бут .756 .742 .650 .709 
without А2 «607 .687 .756 .707 .67o0 .756 .742 .649 -710 
Lawley .608 .687 „755 .7o9 .666 .757 .748 -645 -712 
Centroid 607. .687 .757 .7о5 .673 .755 .741 651 -708 
= 17.8 
ЅАМРІЕ VII 
Variable I 2 3 4 5 6 7 8 Ў 
1 3591 „542 469 .438 .58о .s4r .565 549 
2 -591 :578 573 .583 .616 .548 -553 СЕ 
3 -542 78 от .513 454 458 :530 “57 
4 +409 „573  .sor 515 .583 .522 515 5 " 
5 -438 .583 +513 .515 „519 .495 -500 o 
6 580 .616 .454 .583 .5190 595 +523 497 
7 541.548 „458 522 405 .595 LE ee 
8 *565 7.553 .530 -515 .500 .523 .516 9 
9 "549 .551  .513 .564 .476 .494 .486  .507 
PG 8 
with h? -736 .798 .698 .728 .689 .753 .713 721 -70 
without * ^ .736 .799 .699 .728 .690 .734 713 722 pes 
Lawley :737. .798 .698 .728 бот .756 .714 .729 7 E 
Centroid 736 799 .699 .729 .689 .753 .713 .722 7 


| 
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ЅАМРІЕ VIII 


Variable i 2 3 4 5 


I E 
2 .615 .509 -580 .s61 -467 -540 -517 .522 
з :559 -509 "340 463 -487 -525 -507 +555 
- 4 .576 -580 -549 .631 .560 -595 -574 .618 
5 1544 -561 -463 631 545 -555 -553 533 
6 446 -467 .487 -560 -545 „541 548 -519 
7 «551 540 525 -595 2555 +541 .566  .662 
8 1557 517 -507 574 -553 548 -566 «547 

9 .553 +522 -555 "818 .533 -519 -662 -547 

PC 

with h? глаз -725 +695 797 741 .689 .770 .737 +765 
without Л 743 -726 694 -797 .741 689 .770 -737 765 
Lawley 1742 -726 .694 -796 -742 .689 .771 735 767 
.726 .695 -797 741 1689 .770 -737 -765 


Centroid .743 


SAMPLE ІХ 


5 6 7 8 9 


Variable I 2 3 4 
437 

I .540 574 .566 .448 420 513 512 4 
= +540 483 504 472 406  .500 .466 438 
3 74 483 1479 445 533 491 418 к 
4 .566 .504 -479 459 483 449 45 386 
5 448 472 445 459 424 515 a 15 
6 1420 .406 -533 1483-424 445 m 
7 .513 500 -491 ДО: -а15. «45i КЕШ 12 
8 512 .466 -470 .488 467 417 zu К 
9 .437 432 -529 1505 486 439 483 - 

5 .68 
with Л? узо -686 -727 712 -667 636 E p p 
without A? .730 1686 .726 712 by (ER iG Com 

1688 -726 wir 665 - A ‘Gar “689 


Lawley -733 ane 1667 .635 


Centroid .728 .685 -727 
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SAMPLE X 
Variable 1 % 3 4 5 6 7 8 9 
1 491 .430 .407 .480 .403 .352 .479 -434 
2 491 519 .542 .464 .487 .457 -543 450 
3 -430  .519 505 .441 .442 .415 .459 -520 
4 407 .542 .505 48s .415 .463 .531 475 
8 .480 .464 441 .485 454 .480 .477 -430 
6 доз 487 .442 .415 454 459 .456 431 
ў .352 .457 -415 .463 .480 .459 +550 .430 
8 -479 -543 .459 .531 .477 .456 .550 499 
9 434 450 .520 .475 .430 .431 .430 .490 
PG 
with д2 632 .731 .684 .704 .678 .645 .659 .738 -668 
without Л? 632 .731 .684 .7о5 .678 .645 .659 -737 «669 
Lawley 633 .731 .683 .708 .675 .644 .661 .737 .667 
Centroid 631 .731 .684 .703 .680 .646 .658 .738 -669 
%?=22.1 
SAMPLE XI 
Variable 1 ә 3 4 5 6 7 8 9 
1 .568 .516 .520 .527 .584 .530 -505 504 
2 +568 468 .511 .5о2 .566 .475 -438 460 
3 -516 .468 .500 .510 .492 .512 499 51 
4 -520 .5II .500 .572  .573 8526 -527 594 
5 527.502 .$10 .572 .560 .513 -501 535 
6 584 .566 .492 .573 .560 ‚558 462 +552 
7 "530 .475 .512 .526 .513 .558 525 621 
8 .505 .438 .499 .527 .$от .462 .525 485 
9 -504 .466 .518 .594 .535 .s552 +621 485 
PC 
with A? 436 .687 .69o „узт .73o .757 .739 -676 -742 
without h? 736 .687 .бдо .751 .730 E 738 .676 74? 
Lawley .734 .688 .688 .751 .73o .757 .739 -675 -744 
Centroid 737 .686 .690 .751 .73o .756 .739 -676 742 
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SAMPLE XII 


Variable 1 2 3 4 5 


_ ые ^ SS iae 


1 
2 485 445-400-420 519 428 .473 424 
3 .500  .445 .435 -502 .458 524 507 +500 
4 .486 .400 -435 48s -472 .487 -440 471 
5 .468 .420 .502 485 .561 494 +513 465 
6 .556 -510 .458 -472 561 .5o2 .536 .512 
7 .538 -428  .524 .487 -494 -502 .533 -496 
8 .483 -473 507 440 -513 536 -533 -518 
9 501 424 -500 .471 -465 512 .496 -518 
PC 
with A? .722 -635 „692 .653 -700 .741 720 -720 .696 
without 4° ‚722 :636 -692 1653 701 -741 .720 729 «696 
лох .740 720 -720 .696 


Lawley 


Centroid .722 
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